- users-list - monetdb.org

Re: users-list Digest, Vol 89, Issue 2
by Kalaiarasi Gowtham 13 Jan '20

13 Jan '20

Hi Jenni, Please, find the below steps I have followed to install monetdb *Installation of MonetDB in CentOS 7* *Installing dependencies* # yum update # yum install python autoconf automake libtool bison openssl openssl-devel gettext-devel git # git clone https://github.com/MonetDB/MonetDB.git # cd /MonetDB # ./bootstrap # cd .. # mkdir testdir # cd testdir *To enable py3integration python version 3 should be installed* # yum install centos-release-scl # yum install rh-python36 # scl enable rh-python36 bash *check python version* # python -V then # ../MonetDB/configure –enable-py3integration=yes # make # make install Thanks and regards, Kalai On Wed, 8 Jan 2020 at 16:31, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. Re: pymonetdb issue (ferreira) > 2. Re: Monetdb (Ying Zhang) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 7 Jan 2020 17:22:59 +0000 > From: ferreira <pedro.ferreira(a)monetdbsolutions.com> > To: Communication channel for MonetDB users <users-list(a)monetdb.org>, > ALEX OSSIPOV <alexo15(a)comcast.net> > Subject: Re: pymonetdb issue > Message-ID: > <0e74a920-b2ad-2993-b1dc-fcd7a49105bb(a)monetdbsolutions.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Hello Alex, > > I just looked at pymonetdb source code and indeed it does not support > IPv6 yet. I'm going to create an issue to request for it. The reason is > because in the past, the MAPI protocol (which MonetDB uses for socket > connections) didn't support IPv6, so the clients (Python, Ruby, JDBC, > etc.) never had to implemented them. However they can now add this option. > > Best regards, > > Pedro > > On 12/22/19 5:30 AM, ALEX OSSIPOV wrote: > > Hi All > > I installed pymonetdb v 1.2.1 and trying to connect to remote > > server?MonetDB v11.35.3 (Nov2019) listening on IPV6 only. I have got > > next error in python client: > > >>> mapi_connection.connect(hostname="<server_name>", port=xxxxx, > > username="monetdb", password="monetdb", database="demo", > > language="sql", unix_socket=None, connect_timeout=-1) > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > File "/usr/local/lib/python3.7/site-packages/pymonetdb/mapi.py", line > > 143, in connect > > self.socket.connect((hostname, port)) > > socket.gaierror: [Errno -2] Name or service not known > > >>> > > My pc client is also using ipv6 to connect to MonetDB server. I also > > have jdbc connection that works fine. > > Question: > > Does it mean pymonetdb does not support ipv6 only environment? > > What would be the reason if it has nothing to do with protocol? > > > > Thanks > > Alex > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org > > https://www.monetdb.org/mailman/listinfo/users-list >

2 2

Monetdb
by Kalaiarasi Gowtham 07 Jan '20

07 Jan '20

Hi, Currently, we are trying to start the database created in monetdb using the following command monetdb start <dbname> monetdbd: internal error while starting mserver 'database <dbname> did not initialise the sql scenario', please refer to the logs Getting logs as follows [root@localhost mydbfarm]# tail -f merovingian.log 2020-01-03 16:03:14 MSG merovingian[13157]: database <dbname> has shut down 2020-01-03 16:03:14 ERR control[13157]: !monetdbd: an internal error has occurred 'database <dbname> did not initialise the sql scenario' 2020-01-03 16:03:18 ERR merovingian[13157]: client error: database <dbname> did not initialise the sql scenario 2020-01-03 16:03:18 MSG merovingian[13157]: database <dbname> (13263) has exited with exit status 0 Please,help us to solve this issue. Thanks Kalai

2 1

Announcement: New Nov2019-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 20 Dec '19

20 Dec '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Nov 2019-SP1 bugfix release (11.35.9) Bug Fixes * 6723: columns aliases duplicates should not be allowed. automatic aliasing required. * 6724: Prepare confuses types when more than one argument is used * 6726: Python aggregation does not create aggr_group when aggregating over all rows * 6765: GRANT SELECT privilege on a subset of table columns results in access denied error when selecting the same columns from the table * 6790: Count distinct giving wrong results * 6791: str_to_time('11:40', '%H:%M') creates wrong time value * 6792: JSON path compiler accepts invalid input * 6793: cast(interval second value to int or decimal) is wrong (by a factor of 1000), cast(interval month value to decimal or floating point) fails * 6794: external name fits.listdir not bound (sys.listdir) Fatal error during initialization: * 6796: Incorrect crash time reported by monetdb tool after crash of mserver5 * 6798: json.text off by one error * 6804: DNS resolution of 0.0.0.0 fails on recent Ubuntus

1 0

resetting credentials from mserver5?
by Jonathan Speiser 20 Dec '19

20 Dec '19

Hello, I had tried changing the password on my monetdb server (MonetDB 5 server v11.29.3 "Mar2018") but after rebooting it seems I am locked out from mclient. Is there any way I can change the monetdb user password from mserver5? Thank you for your help Jon

2 1

starting database 'tpch'... FAILED
by DEMBELE Simon Pierre 16 Dec '19

16 Dec '19

-------- Original Message -------- Subject: starting database 'tpch'... FAILED Date: 16/12/2019 12:58 From: DEMBELE Simon Pierre <simon-pierre.dembele(a)ensma.fr> To: mailman-owner(a)monetdb.org Bonjour, Hello I need help; Why can solve this issue starting database 'tpch'... FAILED start: starting 'tpch' failed: database 'tpch' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Thank -- Simon

2 2

MERGE TABLE - CANNOT DROP PARTITION
by Ariel Abadi 10 Dec '19

10 Dec '19

Hi! I'm still having issue dropping internal tables from the MERGE TABLE. I used to have this problem on a previous version. I have created a new DB, imported all the data... and now I need to delete a partition (NOT a table). I'm still facing the same problem, which is that I cannot remove it from the merge table. (as you can see below).. Is there anything I can do? Rgds Ariel Welcome to mclient, the MonetDB/SQL interactive terminal (Apr2019) Database: MonetDB v11.33.3 (Apr2019), 'mapi:monetdb://dt-mn:54321/cli_dg' Type \q to quit, \? for a list of available commands auto commit mode: on sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful sql>ALTER TABLE sb_traf.cub15_cubo DROP TABLE sb_traf.cub8_20190801; operation successful

2 1

CREATE DATABASE
by Ariel Abadi 09 Dec '19

09 Dec '19

Hi! Im trying to start a new fresh database .. I am creating with the following commands: monetdbd create /opt/starbi/mm/ monetdbd start /opt/starbi/mm/ monetdb create cli_dg monetdb start cli_dg When I try to start the new database... the database is NOT starting. starting database 'cli_dg'... FAILED start: starting 'cli_dg' failed: database 'cli_dg' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details *Here is the log:* 2019-12-09 15:59:07 MSG cli_dg[385]: # loading sql script: 72_fits.sql 2019-12-09 15:59:07 ERR cli_dg[385]: #main thread:!ERROR:ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir) 2019-12-09 15:59:07 ERR cli_dg[385]: ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir)Fatal error during initialization: 2019-12-09 15:59:07 ERR cli_dg[385]: ParseException:SQLparser:3F000!CREATE PROCEDURE: external name fits.listdir not bound (sys.listdir) 2019-12-09 15:59:11 MSG merovingian[370]: database 'cli_dg' (385) has exited with exit status 1 2019-12-09 15:59:11 ERR control[370]: (local): failed to fork mserver: database 'cli_dg' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Can you please help!?

4 5

COPY INTO ... LOCKED
by Roberto Cornacchia 02 Dec '19

02 Dec '19

Hi, The documentation about the LOCKED modifier for COPY INTO mentions: *When using the LOCKED mode, it is the responsibility of the database user to make sure that there is currently only a single client connection accessing the database* I was assuming (and the syntax suggests, I think) that the LOCKED refers to the table being copied into, not to the entire database. That is, only that table is updated without the overhead of logging. But the documentation imposes the very restrictive condition that only one connection is active on the database. That seems to suggest that the entire database is being updated without logging, not only that table. Could you please clarify? Can I do COPY INTO LOCKED safely with more than one connection, as long as only 1 connection touches that table? Roberto

1 0

Issue in using Monet/Python loader
by Kalaiarasi Gowtham 29 Nov '19

29 Nov '19

Hi, While trying to use Monet/Python Loader function,we are getting the following error *pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};":str);* *Followed the below steps for installation* *Python installation steps:* We have installed python 3.6 1)yum install autoconf 2)yum install automake 3)yum install libtool 4)yum install openssl 5)yum install openssl-devel 6)yum install python-devel 7)Configured environment path settings for python export PYTHONPATH=$PATH:/usr/local/lib/python3.6/site-packages *Monet installation steps:* 1)git clone https://github.com/MonetDB/MonetDB.git 2)cd /MonetDB 3)./bootstrap 4)cd.. 5)mkdir testdir 6)cd testdir 7)../MonetDB/configure --enable-pyintegration=yes While configuring we get the following *py3integration is enabled* 8)make 9)make install 10)monetdbd create /path/to/mydbfarm 11)monetdbd start /path/to/mydbfarm 12)monet stop -a 13)monet set embedpy=true 14)monet start -a 13)Enabled numpy(pip install numpy) *Created a loader by using the below python script* import pymonetdb import sys import os connection = pymonetdb.connect(username="xyz", password="xyz123", hostname="ipaddress", database="testdb") cursor = connection.cursor() cursor.execute("CREATE LOADER myloader() LANGUAGE PYTHON {_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};") #create loader cursor.execute("COPY LOADER INTO store FROM myloader();") #append the row from loader to table connection.commit() *While running the above script getting the following error* pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status':1})};":str); Please, help us to fix this issue. Thanks & Regards Kalaiarasi A

2 1

Announcement: New Nov2019 Feature release of MonetDB suite
by Sjoerd Mullender 28 Nov '19

28 Nov '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Nov 2019 feature release (11.35.3) MonetDB5 Server * The server "console" has been removed, as has the --daemon option. The server now doesn't read from standard input anymore. The way to stop a server is by sending it a TERM signal (on Linux/Unix) or by sending it an interrupt signal (usually control-C -- on all systems). * Implemented a function bat.diffcand to calculate difference of two candidate lists. * The mtime module was completely rewritten, the atom types date, daytime, and timestamp were changed. Upgrade code for BATs containing these types has been implemented. The old daytime type used a 32 bit integer to record milliseconds since the start of the day. The new daytime type uses a 64 bit integer to record microseconds since the start of the day. The old date type recorded days since or before the year 1. The new daytime type records the day of the month and the number of months since the year -4712 separately in a single 32 bit integer of which only 26 bits are used. Dates now use the proleptic Gregorian calendar, meaning the normal Gregorian callendar has been extended backward, and before the year 1, we have the year 0. Both the old and new timestamp types are a combination of a daytime and a date value, but since those types have changed, the timestamp type has also changed. The new date type has a smaller range than the old. The new date range is from the year -4712 to the year 170049. During conversion of date and timestamp columns, the dates are clamped to this range. * The tzone and rule atom types have been removed. They were not used by any code, and they were defined in a non-portable way. * Added "mapi_ipv6" property to monet_options to force ipv6 address binding only. This property is inherited while forking from monetdbd if it is also set there. * Removed (bat)calc.between_symmetric and changed (bat)calc.between by adding a number of extra arguments, all of type :bit: symmetric, low inclusive, high inclusive, nils false. Merovingian * Added "vmmaxsize" and "memmaxsize" mserver5 options to the daemon in order to set mserver5's maximum virtual and committed memory respectively. * Added ipv6 property to monetdbd properties to force IPv6 addresses binding only. By default this property is false to allow IPv4 addresses as well. MonetDB Common * BATcalcbetween and all its variants now have an extra bool parameter "anti" to invert the test. * All forms of BATcalcbetween and VARcalcbetween have extra bool arguments for low inclusive, high inclusive and nils false. The latter causes the result to be false instead of nil if the value being checked is nil. SQL Frontend * Removed functions json.text(string) returns string and json.text(int) returns string. Their MAL implementation didn't exist, so they were meaningless. * There are new aggregate functions sys.median_avg and sys.quantile_avg that return the interpolated value if the median/quantile doesn't fall exactly on a particular row. These functions always return a value of type DOUBLE and only work for numeric types (various width integers and floating point). * Added sys.deltas ("schema" string, "table" string, "column" string) returns table ("values" bigint) system function which returns a single column with 6 values: a flag indicating if the column's upper table is cleared or not, the count of the RDONLY, RD_INS and RD_UPD_ID deltas of the column itself, the number of deleted values of the column's table, as well as the level of the current transaction in the transaction level tree. It should be used for debugging purposes only. * Added "VALUES row_list" statement as a top SQL projection statement. * The implementation of in-expression now follows a join-based approach instead of using iterative union/selects. This greatly improves performance for large in-value-lists. Furthermore the old approach has large in-value-lists generate large MAL plans. This is now no longer the case. * Strings are now limited to 1GB, double-quoted tokens are limited to 2kB. These sizes are bytes of (UTF-8 encoded) input data. Bug Fixes * 3533: SQL aggregate functions avg(), sum() and median() return an error when used on a column with datatype interval second * 6134: Query produces error: HEAPalloc: Insufficient space for HEAP of 1168033427456 bytes. * 6613: LATERAL crash /.../rel_bin.c:1473: rel2bin_table: Assertion `0' failed. * 6683: Bug in subselect * 6686: Bug in subselect (count function) * 6688: Bug in subselect (or condition) * 6689: Trying to improve the performance of SQL queries with a large list of members in IN clause. * 6695: timestamp transformation * 6700: Monetdb Bugs in Subselect statements: * 6722: window functions issues * 6740: while upgrading the database from verison (MonetDB-11.27.13) to (MonetDB-11.33.3) we are unable to bring up the database * 6754: in mclient a strang msg is reported after issueing command: set schema sys; * 6755: Assertion failure in rel_bin.c * 6756: Error in optimizer garbageCollector on merge tables select * 6757: Double free or corruption (out) * 6758: SIGSEGV in __strcmp_sse2_unaligned() * 6759: Python JSON loader creates invalid data type for strings * 6761: Error: Program contains errors.:(NONE).multiplex * 6762: mserver5 crashes on (re-)start * 6764: mserver5 crashes with corruption, double free, invalid size or invalid pointer * 6766: Missing bulk implementation for get_value and next_value calls * 6769: ProfilerStart is not threadsafe * 6771: R-devel * 6772: TRACE includes information from all active sessions * 6773: json.filter returns corrupted string when selecting JSON null value * 6774: PROD aggregation gives wrong result * 6775: NOT IN with an AND containing an OR gives wrong result * 6776: Creating a table with a full outer join query gives type with wrong digits on the joined key. * 6779: Using Windows Messages translation for errno error codes. * 6780: Wrong value of the rank function * 6781: Insert after index creation crash * 6783: AVG changes scale of its results * 6784: function sys.isauuid(string) should return false if string value cannot be converted to a UUID

1 0

UDF bulk implementation with some arguments as scalars
by Roberto Cornacchia 05 Nov '19

05 Nov '19

Hi, When defining a UDF, e.g. CREATE FUNCTION myfunc(a int, b int) RETURNS int we can have: - a scalar implementation command myfunc(a:int, b:int) :int - a bulk implementation command myfunc(a:bat[:int], b:bat[:int]) :bat[:int] - a mixed implementation (e.g. a is a column, b is scalar): command myfunc(a:bat[:int], b:int) :bat[:int] I'm trying to understand what the restrictions are, if any, for the third case. Is it expected to be allowed in (a combination of) the following cases? - Interleaved bulk/scalar arguments command myfunc(a:bat[:int], b:int, c:bat[:int]) :bat[:int] - Table-returning function command myfunc(a:bat[:int], b:int) (:bat[:int], :bat[:int]) - Table-returning function, used with a sub-select in input SELECT * FROM myfunc( (SELECT a, 10 FROM mytable) ); In particular I can't get the last one to work. Even though the sub-select has a constant as the second column, this constant seems to be first replicated into a full column and then given to myfunc() as a column. When the constant parameters are a few, that becomes a waste of time/space. Plus the annoyance of selecting the first value of a constant column for the actual internal implementation, which of course expects a scalar for those arguments. Am I missing something? Is there a way to make that work as I would expect? Thanks, Roberto

4 5

Report a bug
by lhy1149 04 Nov '19

04 Nov '19

Hello, I don't know if it's appropriate, but because I can't register monetdb's bugzilla, I'm going to report a bug here. This problem causes the insert statement to hang and mserver5 takes up 100% of the CPU time. Here is the code that caused the problem: Create table a(a int, b int, id bigserial); -- comment any of the next 2 lines will resolve the problem Create ordered index a_pk on a(id); Create index a_idx1 on a(a); Insert into a(a) values(1); -- this line will hang Enviroment: macOS Catalina 10.15 MonetDB v11.33.11 I hope the development team can check if there is this problem. Thank you

3 2

MERGE TABLE - CANNOT DROP PARTITION
by Ariel Abadi 28 Oct '19

28 Oct '19

Hi! Can someone help me? Im trying to REMOVE a table from a merge table, when I run the command, it states "operation successful", but the table is NOT removed. Here you can see an example... I have run it several times, but the table is still there :( [image: image.png]

4 7

Guide to installing from source on a FC30 machine with SELinux enabled.
by Arjen P. de Vries 10 Oct '19

10 Oct '19

Dear all, For what it's worth, here a report on building MonetDB from source on a Linux FC30 system with SELinux enabled. Thanks to Sjoerd for some crucial hints in getting it to work, and Roberto for the encouragement to not give up :-) First, build as usual: ./bootstrap ./configure --disable-debug --disable-developer --disable-assert --enable-optimize make -j Now, skip the usual `make install` and build rpm packages instead - this includes a single rpm that applies (almost) all the SELinux policies MonetDB needs. (Missing ones should be reported as bugs.) make rpm Install the created rpm packages in two steps, such that all directories that need policies have indeed been created (the SELinux rpm is under the noarch tree): find rpmbuild/RPMS/`uname -m` -name \*.rpm | xargs sudo dnf --disablerepo="*" install --skip-broken -y find rpmbuild/RPMS/noarch -name \*.rpm | xargs sudo dnf --disablerepo="*" install --skip-broken -y Start the server: sudo systemctl status monetdbd You can always uninstall the rpm packages with some more shell magic (replace x86_64 by `uname -m`): find rpmbuild/RPMS -name \*.rpm | sed -e 's/.*\/$MonetDB.\+fedora30$$\.x86.64\|\.noarch$\?\.rpm/\1/g' | xargs sudo dnf --disablerepo="*" -y rm This was sufficient for my, but if you are struggling with SELinux warnings/errors, it may be useful to know the following basics: You can check assigned filecontexts using ls -alRZ /var/monetdb5 These should align with the file contexts specified in monetdb.fc: grep monetdb5 /usr/share/doc/MonetDB-selinux/monetdb.fc Subsequently install those filecontexts and apply them as follows; e.g., for `.merovingian_properties` you would do: sudo semanage fcontext -a -t monetdbd_etc_t /var/monetdb5/.merovingian_properties sudo restorecon -v /var/monetdb5/.merovingian_properties Remaining issues are the following: + `make -j rpm` does not seem to work correctly. + `MonetDB-client-test` package does not want to install with the above command due to the --disablerepo flag. + SELinux complained about access to resource `cpu` - fixed by using the suggested `audit` in the error message. Good luck, hope it saves some people time! Arjen -- ==================================================================== ICIS, office M1.02.07 Radboud University Mercator 1 Faculty of Science Toernooiveld 212 arjen(a)cs.ru.nl NL-6525 EC Nijmegen, The Netherlands +31-(0)24-365 2354 ===================== http://www.informagus.nl/ ====================

1 0

trace not session-isolated?
by Roberto Cornacchia 09 Oct '19

09 Oct '19

On our production servers we keep a pre-defined number of sessions open to a database. Incoming queries are distributed among these sessions. I create an mclient connection to that database and I run a trace (any query). The trace displays all functions created by the queries belonging to the other sessions. All together concatenated (or maybe interleaved, I'm not sure) in the output of my trace. I expect this is not meant, is it?

1 0

Announcement: New Apr2019 Feature release of MonetDB suite
by Sjoerd Mullender 26 Sep '19

26 Sep '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2019 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Apr 2019 feature release (11.33.3) MonetDB5 Server * Changed the way blobs are ordered. Before, shorter blobs came before longer, now the contents is compared first. * Removed function blob.tostring() since it cannot guarantee that the resulting string is properly encoded in UTF-8. * Removed the type "sqlblob", changed the external format of the type "blob" to be like what "sqlblob" used to be. In other words, the "blob" type is now SQL-compatible. * The MAL functions algebra.sort and algebra.firstn now have a new, extra second-to-last argument, nilslast:bit, which indicates where NIL values are to be sorted: at the beginning, or at the end. The old behavior is when for algebra.sort, the value of nilslast is equal to the value or the reverse:bit parameter, for algebra.firstn when the value of nilslast is opposite to the value of the asc:bit parameter. Build Environment * We now use the C11 atomic operations if available, although libatomic_ops is still used if present and not disabled on the configure command line. * We now also build the Python 3 integration packages for Fedora, Debian, Ubuntu, and Windows (not for Debian 8, Ubuntu 14.04, or EPEL 6 and 7 which are too old). SELinux Support * There is one tunable parameter, mserver5_can_read_home, which can be set using "setsebool -P mserver5_can_read_home=true" to allow an mserver5 process started by monetdbd under the control of systemd to read files in users' home directories. Merovingian * Added a new database property named `profilerbeatfreq`. Its value, <freq>, should be an integer and, if it is set the command `profilerstart` will call stethoscope with this value as the argument to -b. This will instruct stethoscope to collect heartbeat events from the MonetDB server every <freq> milliseconds. Please note that no type checking is done, and if <freq> is not an integer, stethoscope will silently ignore it. Client Package * The mclient program can now be quit by typing quit or exit on the command-line. This makes it easier for novice users to quit the program if they do not know that the standard command to quit is \q. Mapi Library * Removed function mapi_cache_shuffle. * Removed function mapi_stream_query. Stream Library * The functions mnstr_write{Sht,Int,Lng,Hge} and their Array variants now swap bytes of the written values if the stream is set to swap bytes (i.e. big-endian on a little-endian machine and v.v.). * Removed defines ST_ASCII, ST_BIN, ST_READ, ST_WRITE from stream.h. * Changed function mnstr_set_byteorder(stream, char) to mnstr_set_bigendian(stream, bool) where the second argument should be `true' for specifying that the stream is bigendian. This sets an internal flag whether or not to swap bytes which can be retrieved with mnstr_get_swapbytes(stream). * Changed function `int mnstr_byteorder(stream)' to `bool mnstr_get_swapbytes(stream)'; it now returns whether bytes are swapped. * Changed function `int mnstr_type(stream)' to `bool mnstr_isbinary(stream)'. MonetDB Common * During processing, we now try to maintain the smallest and largest values that occur in a BAT. This information is not saved across server restarts. * Added an extra argument of type `bool' to the atom to string and string to atom methods to indicate whether the string is for internal use (casting the value to or from a string) of external use (to be printed). * Implemented a nilslast option for BATfirstn. If set, NILs come last in the ordering that BATfirstn simulates, so non-NIL values are preferentially returned. The old behavior can be obtained by setting nilslast to !asc(ending). * Implemented a nilslast option for BATsort. This option should be equal to the reverse option for stable sort (it is not implemented for stable sort), but can be different from reverse for non-stable sort. The functions BATsort and GDKqsort have extra parameters, the function GDKqsort_rev has been removed (superseded by GDKqsort with the new `reverse' parameter). * The BUNtail, BUNtvar, BUNtloc, and BUNtpos macros (and Tloc and Tpos) now return a `void *' instead of a `char *'. * Function PROPdestroy now takes a BAT* parameter. * A whole bunch of functions now take a bool argument instead of an int argument. Some functions now return a bool instead of an int. In all these cases, the int was used as a Boolean value. SQL Frontend * Implemented two-argument least/greatest functions. If one argument is NULL, the functions return the other value, otherwise they return the least/greatest of the two arguments. * Implemented SQL2003 natural logarithm function: ln(num_expr). * Function octet_length now also accepts a BLOB argument and returns the length of the BLOB (i.e. the same as length when called with a BLOB argument). * Implemented length(blob) and comparison between blobs. * Implemented a two argument function log which calculates the logarithm of the first argument using the second argument as base. * Next to the functions log (natural logarithm) and log10 (base 10 logarithm), there is now also log2 (base 2 logarithm). * Removed unused global SQL variable "history". * Allow usage of custom expressions in GROUP BY and PARTITION BY clauses. The same expressions can be used in the projection phase as long as it is exactly the same used in the grouping clause (textual lookup is performed). As an example the query: SELECT col1*2 FROM t1 GROUP BY col1*2; is correct, while SELECT sin(col1+5) FROM t1 GROUP BY col1*2; is wrong. * Allow usage of WITH clauses in insert, update, delete and merge statements. * Implemented merge statements from SQL:2003 standard. Using a source relation R, a target table T is updating depending on the result of the merge. In case of a match, the table T's row is either updated or deleted with R's row. In a non-match case, R's row is inserted into T. The grammar is the follows: > MERGE INTO target [ [AS] ident ] USING source ON search_condition { WHEN MATCHED [ AND search_condition ] THEN { UPDATE SET assignment_list | DELETE } } | { WHEN NOT MATCHED [ AND search_condition ] THEN INSERT [ column_list ] [ { DEFAULT VALUES | VALUES row_values } ] } * Added possibility to change the schema of a table with the syntax: > ALTER TABLE [ IF EXISTS ] qname SET SCHEMA ident * Added optional alias option for target table in update and delete statements: (e.g. UPDATE target AS alias SET col1=1 WHERE alias.col2 < 5) * Improved and extended storagemodel functionality. Changes include: * Added views sys."tablestorage" and sys."schemastorage" for easy and quick assesment of aggregated storage usage per table or per schema. * Excluded listing system tables in views sys."storage", sys."tablestorage" and sys."schemastorage". You can still use sys."storage"() to query storage of system tables and columns. * In procedure sys.storagemodelinit() when populating table sys.storagemodelinput it now excludes system tables as system tables are not useful to be modeled for storagesize by application users. It now also computes and populates the atomwidth column more correctly for variable size data types (strings, blobs) when the table has representative data in those columns. * System function sys.columnsize() has been corrected for types: tinyint, real, date, time, timetz, sec_interval, month_interval, decimal, uuid, mbr, char, clob, json, url, blob, geometry and geometrya. For variable size data types (varchar, char, clob, json, url, blob, geometry, geometrya) it now returns the columnsize excluding the variable heapsize. The heapsize is retrievable via sys.heapsize(). * System function sys.heapsize() has been corrected for all variable size data types: varchar, char, clob, json, url, blob, geometry and geometrya. * System function sys.imprintsize() has been corrected. The imprintsize depends on the width (1 or 2 or 4 or 8 or 16 bytes) of the data type. Also instead of 12% it now uses 20%, which is a better estimate. * System function sys.storagemodel() has been removed as it outputs the same data as view sys.storagemodel. Use view sys.storagemodel instead. * Corrected views sys.storagemodel and sys.tablestoragemodel by returning a computed orderidxsize when the count has been changed in the sys.storagemodelinput table. Also the views now return data ordered by schema, table and column names. * Extended view sys.tablestoragemodel with column: "storages". Besides columns also keys (primary, foreign and unique keys) and indexes (ordered, imprints) use storage, so the "storages" count can be higher than the number of columns per table. * Corrected the data type of columns "schema", "table", "column", "type", "mode" and location in table sys.storagemodelinput and functions sys."storage"(), sys."storage"(sname), sys."storage"(sname, tname) and sys."storage"(sname, tname, cname) from string into varchar(1024). Consequently also the views based on the table or functions will be reporting varchar(1024) as column meta data instead of clob for those columns. This allows faster querying and reporting by generic SQL programs which treat clob querying different from varchar columns. * Extended windowing functions catalog with SQL standard semantics. Standard aggregation functions can now be used in windowing functions: (avg,max,min,sum,prod,count). Other windowing specific functions were also implemented: (percent_rank,cume_dist,ntile, lag,lead,first_value,last_value,nth_value). * The standard frame specification was implemented for aggregation functions as well as first_value, last_value and nth_value functions. The available frames are rows, range and groups. Bounds can be unbounded (partition limit), the current row, a fixed number of rows (constant), or variable (column as input). (e.g SELECT COUNT(col1) OVER (PARTITION BY col2 ORDER BY col3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) FROM t1;). * Added WINDOW keyword which is optionally provided after the FROM clause, with window specifications used in the projection (e.g SELECT SUM(col1) OVER w1, LAST_VALUE(col2) OVER w2 FROM t1 WINDOW w1 AS (ROWS BETWEEN 5 PRECEDING AND 0 FOLLOWING), w2 AS (w1);). * Our previous partitioning implementation didn’t impose order in the input. With this reexamination, partitioning now imposes ascending order by default, thus pairing with the industry standard implementation. * Implemented X'...' style binary string literals. * Implemented U&'...' Unicode character string literals and U&"..." Unicode delimited identifiers, including UESCAPE. For the string literals, you can have U&'...' '...' '...' UESCAPE '...' where the escape must be as single character and the other '...' strings are also Unicode character string literals. For now, these latter strings also undergo C-style backslash interpretation. * Implemented PostgreSQL-like E'...' strings. The strings can contain C-style backslash escapes. The old format strings '...' currently still also accept C-style escapes, but that feature will be removed in a future release. * Implemented the NULLS FIRST and NULLS LAST option to ORDER BY. The default is NULLS FIRST for ASC(ending) and NULLS LAST for DESC(ending). * Added possibility to rename SQL schemas, tables and columns with ALTER statements: > ALTER SCHEMA [ IF EXISTS ] ident RENAME TO ident > ALTER TABLE [ IF EXISTS ] qname RENAME TO ident > ALTER TABLE [ IF EXISTS ] qname RENAME [ COLUMN ] ident TO ident Also added optional IF EXISTS clause to other existing ALTER TABLE statements. * The sql SAMPLE syntax is extended to include an optional integral SEED parameter that sets the seed for the internal random number generator in the sample algorithm. When the seed is set, the user can obtain a reproducible sample set from a static dataset with the same sample query. The new syntax is SELECT ... FROM ... WHERE ... SAMPLE <expr> [SEED <integer>] * Implemented a version of COPY INTO that reads/writes files from/to the client instead of doing it in the server. This has the advantage that COPY INTO is then no longer restricted to only the "super user" monetdb, nor only to absolute file names. The syntax to have the server communicate with the client for file content is COPY INTO table FROM file ON CLIENT ...; and COPY query INTO file ON CLIENT ...;. This also works for COPY BINARY INTO. There is also the possibility to specify that files are to be read/written by the server by using ON SERVER. This is also the default when ON CLIENT or ON SERVER is not specified. * The sys.functions table now has an extra column called "system" of type BOOLEAN whose value is TRUE for system functions (i.e. functions that should not be dumped). The table sys.systemfunctions has been changed to a view and is now officially deprecated. * Extended merge tables with partitioning using a predicate: > CREATE MERGE TABLE [ IF NOT EXISTS ] table_name (... columns ...) [ PARTITION BY { RANGE | VALUES } { ON '(' column_name ')' | USING '(' expression ')' } ] The partitioning can occur by range or list of values using one of the table's columns or an expression. The domain of the partitioning scheme will be covered by each child table added with alter statements depending if the table is partitioned by range or list of values: > ALTER TABLE merge_table_name { ADD | SET } TABLE child_table_name AS PARTITION IN '(' expression [ ',' ... ] ')' [ WITH NULL ] > ALTER TABLE merge_table_name { ADD | SET } TABLE child_table_name AS PARTITION BETWEEN { RANGE MINVALUE | expression } AND { RANGE MAXVALUE | expression } [ WITH NULL ] > ALTER TABLE merge_table_name { ADD | SET } TABLE child_table_name AS PARTITION WITH NULL When updating a single partition range/list of values, the SET keyword should be used instead of ADD. INSERT, UPDATE and DELETE statements are possible on partitioned tables with corresponding validation on each partition domain. The system tables sys.table_partitions, sys.range_partitions and sys.value_partitions were added to store meta-information about each existing partition. Bug Fixes * 2403: stream: http read support in streams * 2416: GDK: file/dir creation mask * 2478: SQL: rename columns * 2496: SQL: implement greatest / least * 3384: Auxiliary window functions * 3416: log() SQL function only supports natural logarithm * 3448: Make lbatomic_ops optional if installed * 3520: Add support for scalar function LN(num) * 3530: sigabort on window aggr function * 3567: Add support for: ALTER TABLE [schema1.]oldtablename RENAME [TO] [schema2.]newtablename * 3743: DELETE FROM merge table not supported. * 3832: Cannot use expressions in GROUP BY clause * 3946: Expose C log2 function to SQL * 4056: mapi_timeout is not implemented * 6181: comments cause error with return in function * 6326: Eliminate unused UNION tables * 6347: Mserver returns with "memory exhausted in" error if query size exceeds 2.5MB * 6475: Remove unused SQL global variable "history" * 6588: Table aliasing are not supported in SQL update queries * 6591: Name mangling in the SQL catalog results in ambiguity * 6608: Sqlitelogictest error message in group by with coalesce * 6609: Rename Schema in MonetDB * 6636: sys.queue() extension: allow DBA to see all running queries * 6647: Add suport to Python 3 on Windows * 6669: COPY [xxx RECORDS] INTO foo FROM STDIN ... doesn't work without specifying nr of to be copied records * 6671: Error when running user function on merge table with remote part * 6674: R UDF with Date type fails to convert RAPI * 6676: Max data length of 2048 for column sys._tables.query is too small for the actual data (2811 for view sys.ids) * 6678: Binding NULL parameter to parametrized query results in syntax error on execution * 6680: Copy cannot open CSV file if systemctl enabled * 6684: Inserting into a replica table crashes the server * 6685: adding a view to a merge table gives unexpected behaviour * 6690: Unable to fetch empty string with ODBC driver * 6696: Re-use of MAL explain plans with complex type coercions

2 1

stethoscope not redirecting time column correctly
by Jiading Guo 17 Sep '19

17 Sep '19

Hi, When setting stethoscope dumping to disk by `stethoscope -d <db-name> -o <output-file>`, the time column is printed in the console instead of the file. In `MonetDB/clients/mapiclient/stethoscope.c` L113, it directly prints to stdout by `printf("\"%s\", ", ev->time);`. Is it on purpose? Also, I can't follow this message[1] to profile MAL because it seems that the APIs have been changed. Where can I find the newest document? Thanks, Jiading Guo [1]: https://www.monetdb.org/pipermail/users-list/2013-May/006369.html

2 2

!SQLException:sql.bind:42000!Column type mismatch
by Jiading Guo 09 Sep '19

09 Sep '19

Hi, I'm doing some tests about bind operation with v11.33.11 (Apr2019-SP1) and here's how to reproduce: ``` mclient -d test sql>create table unary(a int); sql>insert into unary(a) select value from generate_series(1, 101, 1); mclient -d test -l mal mal>sql.init(); mal>s := sql.mvc(); mal>a := sql.bind(s, "sys", "unary", "a", 0); ``` Then I got `!SQLException:sql.bind:42000!Column type mismatch`. I traced it down to `sql/backends/monet5/sql.c:mvc_bind_wrap`, `int coltype = getBatType(getArgType(mb, pci, 0));` coltype is set to 255 here, which confused me about what this coltype is used for. Also in `sql/backends/monet5/sql.mal`, the comment says the access kind (last argument) is divided into `0, base table`, `1, inserts` and `2, updates`. In what scenarios are these values used? Is `0` for read only operations? What about delete operations? Thanks for your help. Best regards, Jiading Guo

2 2

Latest release (11.33.11) still doesn't honor numactl
by Gotwisner, Dave 04 Sep '19

04 Sep '19

I have just upgraded MonetDB on an 88 core CentOS box - 2 cpu modules. I was also one of the people who originally found the numa issue. The system doesn't behave as I would expect. Here is the sequence of commands I execute to start up MonetDB (overkill, but creating a clean database for the test): 1. numactl --cpunodebind=1 --membind=1 monetdbd create /srv/zfsdata/monetdbTesting/nTC 2. numactl --cpunodebind=1 --membind=1 monetdbd set listenaddr=0.0.0.0 /srv/zfsdata/monetdbTesting/nTC 3. numactl --cpunodebind=1 --membind=1 monetdbd set discovery=false /srv/zfsdata/monetdbTesting/nTC 4. numactl --cpunodebind=1 --membind=1 monetdbd set control=false /srv/zfsdata/monetdbTesting/nTC 5. numactl --cpunodebind=1 --membind=1 monetdbd start /srv/zfsdata/monetdbTesting/nTC 6. numactl --cpunodebind=1 --membind=1 monetdb -p 50000 create nTC 7. numactl --cpunodebind=1 --membind=1 monetdb -p 50000 release nTC If I launch MonetDB using the monetdbd command as "numactl -cpunodebind=1 -membind=1", that means that we are assigning only 44 cores for use with MonetDB (mserver5). I would expect that either command 1 or 5 is the one that spawns mserver5. What I then see with numactl --show --pid=`pidof mserver5` is: policy: default preferred node: current physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 cpubind: 0 1 nodebind: 0 1 membind: 0 1 I would have expected that if we launch mserver5 via the monetdbd wrapper (as is what's documented in the tutorials), that the numa settings would be honored. Note also, I would expect 44 threads (plus a few extra) be set rather than 88 (plus a few extra). We are seeing 96 threads. Looking at the useful part of the /proc/`pidof mserver5`/status file gives the following: Cpus_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00fffffc,00000fff,ffc00000 Cpus_allowed_list: 22-43,66-87 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 Mems_allowed_list: 0-1 Which implies it IS putting the program on 44 cores, but still creating 96 threads. It also shows that it isn't honoring the memory constraint? Any suggestion on how to correctly launch so it honors our numa requirements? Thanks, Dave

2 1

Possible problem with C++ based CREATE FUNCTION
by Gotwisner, Dave 04 Sep '19

04 Sep '19

I am trying to create a function that works around concurrency issues that are documented, and that we are seeing, when doing simultaneous "create table" statements and bulk insert statements (or selects) off of other tables. To do this, I am implementing a function that uses a boost::shared_mutex that we call around transactions. We are using a custom version of the boost library, so it is installed in a non-standard place: /usr/local/boost-1.54.0 for development, and /opt/xxx/ for production. >From the documentation, it appears that I can use #pragma LDFLAGS and CFLAGS to point to the non-standard directories. It appears to honor the CFLAGS option, but not the LDFLAGS option. If I use the standard /usr/lib64 libraries, calling my function works (but with the wrong library). Adding the #pragma in, I get the following link error: Failed to open shared library: libboost_thread.so.1.54.0: cannot open shared object file: No such file or directory. If I modify LD_LIBRARY_PATH before starting MonetDB, the problem goes away. Ideally, we don't want to have to add all of our relevant directories into the environment, but would expect the #pragma to work. Here is the function definition: CREATE FUNCTION rwlocker(input INT) RETURNS INT LANGUAGE CPP { #pragma LDFLAGS -L/usr/local/boost-1.54.0/lib -lboost_thread #pragma CFLAGS -I/opt/tc3/include -I/usr/local/boost-1.54.0/include #include <boost/thread/shared_mutex.hpp> // use lock() for schema changes, lock_shared() for inserts and queries static boost::shared_mutex readerWriterMutex; result->initialize(result, 1); int val = input.data[0]; switch(val) { case 1: // shared lock readerWriterMutex.lock_shared(); break; case 2: // shared unlock readerWriterMutex.unlock_shared(); break; case 3: // unique lock readerWriterMutex.lock(); break; case 4: // unique unlock readerWriterMutex.unlock(); break; default: // Error break; } result->data[0] = val; }; Invocation is easiest with: select rwlocker(12); which will just output a 12, bypassing the reader/writer locking. Any thoughts on why #pragma LDFLAGS doesn't appear to work right? We are using 11.33.3 on CentOS 7.3.1611 Thanks, Dave

2 2

Announcement: New Apr2019-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 02 Sep '19

02 Sep '19

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2019-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Apr 2019-SP1 bugfix release (11.33.11) Build Environment * Removed restriction on using combinations of --enable-assert, --enable-debug, and --enable-optimize. --enable-debug adds a -g (or -even -g3) option, --enable-debug=gdb adds a -ggdb3 flag (for GCC); --enable-optimize adds a bunch of optimization flags; --disable-debug (or --enable-debug=no) removes any -g flags; --disable-optimize removes any -O flags; --enable-optimize=auto leaves the optimization flags untouched. MonetDB Common * We now look at the limits imposed by cgroups and the setrlimit system call to initialize some internal values related to how much (virtual) memory we think is available. SQL Frontend * Many fixes for using the server in a highly concurrent setting with many clients doing read-only queries mixed with clients doing updating queries. * Changed the internal representation of RANGE MINVALUE and RANGE MAXVALUE in MERGE TABLE partitions. Before, the limits were represented by the smallest and largest value of the domain of the column, now these are represented by a NULL value. This has the added benefit that if you use TO RANGE MAXVALUE, the largest value of the domain is included in the partition (before it wasn't). Also, these new limits now work for any ordered type, so including VARCHAR. Bug Fixes * 6576: Sqlitelogictest aritmetic expressions with negative numbers handling * 6697: Duplicate expressions not eliminated with long CASE statement * 6701: When changing the schema name of a table, referencing rows from sys.columns, sys.keys and more tables are not removed * 6703: SQL optimizer enters loop and goes into stack overflow * 6706: prepare doesn't recognize merge statement * 6712: Where clause with cast ignores sub-select * 6713: COPY INTO FROM 'file.xz' does not work * 6714: Assertion failure in rel_select.c for correlated subquery in aggregation query (with group by and having and order by and limit) * 6715: Assertion failure in rel_bin.c for MERGE INTO command * 6716: COPY INTO does not load UTF8 encoded text * 6718: Can't copy table into another table with constraints (mkey.bulk_rotate_xor_hash) * 6720: Compilation issues on gcc 9.1.1 [-Werror=stringop-truncation] * 6721: Possibly incorrect call to pcre_exec * 6725: Prepare statement on non-existing function crashes * 6728: SELECT DISTINCT not removing duplicates * 6729: Creating a table with duplicate column name should not be possible * 6730: sys.stop or sys.pause a INSERT query triggers "BATproject: does not match always" * 6736: Unexpected query result with merge tables and view after upgrade * 6738: issue with window functions and hugeint type coercion * 6741: report no such table when execute “alter table tb1 rename to tb2” according to using mapi * 6744: CREATE SEQUENCE seq; fails * 6748: MonetDB "forgets" columns after ALTER TABLE statements * 6749: mserver5 restart aborts/segfaults after dropping column with constraint * 6751: ODBC driver(11.33.3) Seg Faults when "fn ucase" is used in SQL

1 0

Crash - !mvc_init: unable to create system tables
by Nobody Noone 30 Aug '19

30 Aug '19

Hi, Would anyone have any advice on recovering. I was running a bunch of COPY SELECT * FROM x INTO '/y.z.csv' USING DELIMITERS ',' , '\n' , '"'; The log said there was already an existing file in the dir. The db was not responding so I stopped it. Then tried to start it and I get; !mvc_init: unable to create system tables #main thread:!ERROR:SQLException:SQLinit:42000!Catalogue initialization failed Fatal error during initialization: SQLException:SQLinit:42000!Catalogue initialization failed # MonetDB 5 server v11.33.3 (Apr2019) database 'y' (53342) has exited with exit status 1 ERR control[33899]: (local): failed to fork mserver: database 'y' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Any advice on how to recover? Thanks:)

1 0

Performance issue related to dropping (empty) tables
by Gotwisner, Dave 27 Aug '19

27 Aug '19

I am seeing a performance issue related to schema upgrades. This is from a C++ program that uses the MAPI library. We try and keep 30 days worth of data, imported at a fairly high data rate with lots of columns. We use multiple threads to ingest data (bulk insert), and the number of threads is stored in a configuration table. For 20 days worth of data, this works out to around 9000 tables (plus merge tables). We have roughly 6 sets of tables per hour, each of which has 4 tables and a merge table connecting them. Because this is a schema change, we wrap the entire upgrade in a transaction. For the purpose of the test, we had no data in the tables, but only had empty tables. If we change our setting from 4 parallel tables to 2, we end up dropping 9000 tables. About half of the "drop table" commands take 0 seconds. The rest take anywhere between 4 and 12 seconds each. Here is a plot of the time per delete, y access is seconds, x is the statement. The X access is NOT time based: [cid:image001.png@01D5568E.A5389030] Any idea on how we can reduce the time? The total time to delete 9000 tables was 480 minutes. Thanks, Dave

2 4

memory control on large results
by Daniel Zvinca 15 Aug '19

15 Aug '19

Hello, I plan to use the embedded library (custom build) for a project that will likely address very large tables. While the control over statements is handled to the end user, I need an advice how to control memory usage on fetching huge amount of data. Specifically, I already skipped the default "monetdb_result_fetch" and implemented a compressed storage (which optional can be stored on disk), but I wasn't able to see a way of releasing the associated memory for already processed columns (persistent or transient bat). Unless the bat contents is shared between different res_col's, I would like to be able to release that memory to the system before proceeding to the next column. Any suggestion? Thank you, Dan

1 0

Embedded Python Initialization Issue
by Sharma, Sreejith 14 Aug '19

14 Aug '19

All, I am using an Python based UDF and have enabled embedpy=true for DB with Python version 2.7.5. This works fine with CentOS 7, but the same doesn't wok on Red Hat Ent Linux version. Is there any dependency on this? Is it due to OS version? Am getting below error though Python path is proper in Red Hat OS version server. Embedded Python is enabled but an error was thrown during initialization. Can anyone provide some help on this? Both uses same version of MonetDB server version. MonetDB server on which function works fine [user@lnx1535 ~]$ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" MonetDB server on Azure where function is not working [user@alcazimdbd01 ~]$ cat /etc/os-release NAME="Red Hat Enterprise Linux Server" VERSION="7.5 (Maipo)" ID="rhel" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="7.5" PRETTY_NAME="Red Hat Enterprise Linux" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server" HOME_URL="https://www.redhat.com/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" REDHAT_BUGZILLA_PRODUCT_VERSION=7.5 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="7.5" Regards, Sreejith

2 1