- users-list - monetdb.org

Database crash issue
by Amit Ambekar 12 Apr '19

12 Apr '19

Hi, I am using MonetDB from last 2 years, but never faced such issue till now. I am not able to use database ,it is giving error as below: monetdbd: internal error while starting mserver database has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details' Regards, Amit Ambekar Data Engineer

2 2

Get stuck when trying to copy large amounts of data
by Yinjie Lin 02 Apr '19

02 Apr '19

Hi, I'm trying to copy large amounts of data into MonetDB, but the processes would get stuck in the middle. The following is what I did. I have 6 tables, all merge tables partitioned by range of time. And there are hundreds of data files, each has almost 10 million rows and is of size about 3.5 GB. For each table, I use the following command to copy data: `cat file.txt | xargs -n1 -P1 -I {} sh -c "mclient -p 50000 -s \"copy 10000000 records into tbx from stdin best effort\" - < '{}'" ` where file.txt contains paths of files to be loaded. So 6 mclient connections are created to copy data in parallel. But mclient processes would get stuck after loading more than 200 million rows each table. Then I have to kill the processes and restart the database. The environment is CentOS 7.3, 512 GB RAM. MonetDB is compiled using the latest master branch. So does anyone know why this happened? Does MonetDB have a limit on the size of data one instance can hold? Or if there is something wrong with my practice? Thanks in advance! Yinjie Lin

2 1

Performance issue with count distinct
by AUBERT Simon 29 Mar '19

29 Mar '19

Hello all, Here a simple issue I have with performance : 1/ I create a table A and populate it with a bulk load 2/I create a table B, I populate the table with bulk load 3/I create table C with a CTAS statement. I analyze the statistics and set the primary key 4/I try a "count distinct" type query (on all the table) on the PK Field. It is pretty slow, with a high sollicitation of my hard drive for several dozens of second. the "select count" is however fast. So I have these following questions : 1/why despite statistics the count distinct query is slow? I thought the idea of statistics was to get some metrics about the table? 2/if the field I make a count distinct on is the PK (or the set of field is the PK), it's exactly the same than a count non null. So why the time are not the same? 3/if these issues are design issue or bugs, how can I ask to make it solved? For information : Monetdb 11.31.13 on Windows 10 Best regards, Simon AUBERT [cid:691B9F4089B9F34B8E0C22B39B7117EC@eurprd07.prod.outlook.com] Bourse Maritime - 1, place Lainé - 33 000 Bordeaux Mob : +33(0)6 66 28 52 04

2 1

Program contains errors.:(NONE).multiplex
by Niklas B 29 Mar '19

29 Mar '19

Hi, I'm upgrading from an old MonetDB (11.26.0 source snapshot) to the latest proper release. However, my Python aggregate functions appear to have stopped working (Program contains errors.:(NONE).multiplex): Running mserver5 with: /usr/local/bin/mserver5 --daemon=yes --set embedded_py=true --dbpath=/data/monetdb-farm/ --set gdk_nr_threads=1 --set max_clients=6 Reproduce with: CREATE FUNCTION test_udf("conum" int) RETURNS STRING LANGUAGE PYTHON3 {{ }}; Create table test("id" uuid, "date" DATE, "number" INT); select test_udf("number") from test; Gives: Program contains errors.:(NONE).multiplex I'm running it via Docker (stock Debian) with only MonetDB installed via apt-get. Not sure if this is connected: https://www.monetdb.org/bugzilla/show_bug.cgi?id=6378 <https://www.monetdb.org/bugzilla/show_bug.cgi?id=6378> Anyone having any similar issues? Tried compiling from source (April 2019 bransch) as well, no luck Regards, Niklas

3 2

sys.storage sorted attribute changes for primary key
by Jeremy Norris 29 Mar '19

29 Mar '19

Hello, I have a large table (~700 million rows, ~150 columns). Initially after import, the database is very fast on primary key lookups, however after a few thousand creates/updates/deletes on the database, "sorted" (as reported by sys.storage), flips from true to false. This results in major performance degradation. What are the conditions under which a database can lose it's sorted-ness on the primary key column? eg: sql>select schema, table, column, sorted from sys.storage where schema = 'test1' and table = 'cmdb' and column = 'pk_id'; +---------+-------+--------+--------+ | schema | table | column | sorted | +=========+=======+========+========+ | test1 | cmdb | pk_id | false | +---------+-------+--------+--------+ Thanks, -Jeremy Norris

2 1

Fwd: Question about partition bounds of updatable merge tables
by Yinjie Lin 06 Mar '19

06 Mar '19

Hi, I found that there is a new feature called updatable merge table in the new release (Updatable merge tables with partition predicates <https://www.monetdb.org/blog/updatable-merge-tables>). I think this makes merge table similar to *partitioned table* in other databases. But I found that the upper bound of range partitioning is an inclusive bound, while the upper bound of some databases like MySQL and PostgreSQL is an exclusive bound. So my question is: why is the upper bound declaration designed like this? Will it be possible if MonetDB be consistent with other databases? Besides, currently merge table doesn't support partition pruning, so all child tables will be scanned even if the filter condition contains the partitioning key. Is it possible if this feature will be added one day? Best Regards, Yinjie Lin

1 0

#!ERROR: GDKmove: cannot rename /mnt/g/monet/wal/sql/log to /mnt/g/monet/wal/sql/log.bak
by Michael Jansen 21 Feb '19

21 Feb '19

Hello, I have a crash after about 5 minutes when running in transaction replication mode. After the crash I cannot access the db. I am running: mserver5 --trace --dbpath=/data/monetdb/dbfarm/test_db/ --set gdk_keep_persisted_log_files=1000 --set gdk_logdir=/mnt/g/monet/wal --set sql_debug=4 I am using GlusterFS under /mnt/g to manage the replication. I am running on AWS Linux 2. and I have free space on in both the data and wal directories. (I assume this is enough space): Filesystem Size Used Avail Use% Mounted on 10.0.1.68:/gv0 8.0G 7.4G 669M 92% /mnt/g bash-4.2$ the db only has one test table: sql>create table t (n float, t text) ; operation successful sql>select * from t ; +--------------------------+------+ | n | t | +==========================+======+ | 1.1 | blah | | 1.1 | blah | | 1 | blah | +--------------------------+------+ 3 tuples any help is appreciated! -mj

1 0

Error when installing on Centos 7
by Trevor Andreas 25 Jan '19

25 Jan '19

Hello, I am getting an error trying to install MonetDB following the instructions here: https://dev.monetdb.org/downloads/epel/ The error I am getting is when running: yum install -y MonetDB-SQL-server5 MonetDB-client failure: repodata/repomd.xml from monetdb: [Errno 256] No more mirrors to try. https://dev.monetdb.org/downloads/epel/7/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found Looks like the repodata is missing for Centos 7? It looks to be there for Centos 6 https://dev.monetdb.org/downloads/epel/6/x86_64/repodata/ Thanks! Trevor

2 1

Invalid Credentials error
by AUBERT Simon 17 Jan '19

17 Jan '19

Good morning, I have used Monetdb for a few weeks now and this evening I cannot login after a full system restart (windows+monetdb). Environment : Windows 10 (well, it's my work computer, I don't have a lot of choice) Monetdb 5 server v11.31.11 I start monetdb by using C:\Program Files\MonetDB\MonetDB5\bin\mserver5.exe Then mclient on C:\Program Files\MonetDB\MonetDB5\mclient.bat (and usually it work just fine) I also tried with mclient.exe using the -x command to debug. Error message : C:\Program Files\MonetDB\MonetDB5\bin>mclient -X -d demo user(win32):monetdb password: MAPI = monetdb@localhost:50000 ACTION= mapi_reconnect ERROR = !InvalidCredentialsException:checkCredentials:invalid credentials for user 'monetdb' Of course, I use the default monetdb/monetdb credentials. I have no idea of what happens and moreover I have lost an entire schema (I see that if I connect through Tableau Desktop...). I have done an upgrade on the last version (11.31.13) and same result, impossible to connect. And same result. I spend the entire evening on that a Thank in advance. Simon AUBERT [cid:691B9F4089B9F34B8E0C22B39B7117EC@eurprd07.prod.outlook.com] Bourse Maritime - 1, place Lainé - 33 000 Bordeaux Mob : +33(0)6 66 28 52 04

2 3

table-returning function with subselect in input - no mitosis?
by Roberto Cornacchia 14 Jan '19

14 Jan '19

Hi, * Background: I have a custom string tokenizer that I invoke this way: SELECT * FROM tokenize( (SELECT id, s FROM my_strings) ); The output has of course a 1-to-N rows relation with the input. The function has a bulk implementation, so it takes M rows and it outputs M rows. * The question: No matter how large the input table my_strings is, mitosis does not seem to kick in. Is it not allowed in this case because it cannot assume that my custom function allows to concatenate partial results? Do you see any chance for me to enable mitosis with this scenario? Thanks, Roberto

1 1