- users-list - monetdb.org

Re: grafana-monetdb-datasource installation problem
by Svetlin 03 Dec '18

03 Dec '18

Hi Ariel, The current version of the grafana-monetdb-datasource plugin cannot be used with the latest grafana release. There is a missing breaking dependency that has being introduced(see #10053 <https://github.com/grafana/grafana/issues/10053>) and we didn’t have time to patch that in. Secondly, you would need a http server as a mediator to forward the queries to monetdb. The grafana monetdb datasource plugin merely creates MonetDB compatible queries and forwards responses back to grafana core for visualization. I hope this helps. Cheers, Svetlin > On Dec 3, 2018, at 11:03 AM, Ying (Jennie) Zhang <ying.zhang(a)monetdbsolutions.com> wrote: > >> aabadi(a)starconnecting.com <mailto:aabadi@starconnecting.com>

2 1

grafana-monetdb-datasource installation problem
by Ariel Abadi 01 Dec '18

01 Dec '18

Does anybody has installed the monetdb datasource plugin for Grafana? Im running Grafana v5.3.4. The plugin was downloaded from https://github.com/MonetDBSolutions/grafana-monetdb-datasource And is not working... :'( Does anybody knows how to connect MonetDB within Grafana? Any ideas will be very much appreciated!!! Thks Ariel

2 1

Re: Issues trying to create a table...
by Stefan Manegold 28 Nov '18

28 Nov '18

Hi, Internally (as in in the MonetDB code base),the impact of the problem could imho be significantly reduced (though not fully avoided)by "simply" using different separator characters (or even strings?) when creating qnames,e.g. schema:table;column insteadof schema_table_column ... A unique bijective reversible mapping from schema,table,column to qname should avoid the problem entirely , but might require to use separator chracter(s) that are not valid in SQL identifiers, if such exist ... Best,Stefan -------- Original message --------From: Ying Zhang <Y.Zhang(a)cwi.nl> Date: 28/11/2018 07:33 (GMT+01:00) To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: Issues trying to create a table... > On 28 Nov 2018, at 04:18, Herman Knief <hknief(a)gmail.com> wrote: > > Hi Folks, > I'm new to MonetDB and trying to setup a schema from a MariaDB export. > I have a particular table that reports a conflict with the create statement. I've stripped this down to just a particular field that seems to be causing the issue, but unsure where to go from here. > > sql>CREATE TABLE "sc_cat_item_delivery_plan" ("script" string); > CREATE TABLE: zboot_sc_cat_item_delivery_plan_script conflicts Hai Herman, This is because something has already been defined before, whose complete qname is “zboot_sc_cat_item_delivery_plan_script”. This often happens with (but probably not limited to) schema_name+table_name+column_name. In MonetDB test suite, we have two examples to trigger this (type of) error: sql>create table foo (id integer, bar_id integer); operation successful sql>create table foo_bar (id integer); CREATE TABLE: sys_foo_bar_id conflicts and sql>create table foo_bar (baz int); operation successful sql>create table foo (bar_baz int); CREATE TABLE: sys_foo_bar_baz conflicts This is not a bug. You need to change your table and/or column definitions, like your alternatives below. Then CREATE TABLE will succeed. Regards, Jennie > > What's interesting is that either changing the name of the table or the name of the field and these statements work. There is something about this particular combo that is problematic. > sql>CREATE TABLE "sc_cat_item_delivery_plan1" ("script" string); > operation successful > sql>CREATE TABLE "sc_cat_item_delivery_plan" ("script1" string); > operation successful > > I have a total of 2251 tables... all were created except for this one. I've tried to destroy and recreate the schema with no luck. Logs don't seem to indicate any issue and trying to use the debug on the client didn't get me anywhere. Thoughts? Suggestions? > > -- > Please excuse typos... I can't type. > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

2 1

Issues trying to create a table...
by Herman Knief 28 Nov '18

28 Nov '18

Hi Folks, I'm new to MonetDB and trying to setup a schema from a MariaDB export. I have a particular table that reports a conflict with the create statement. I've stripped this down to just a particular field that seems to be causing the issue, but unsure where to go from here. sql>CREATE TABLE "sc_cat_item_delivery_plan" ("script" string); CREATE TABLE: zboot_sc_cat_item_delivery_plan_script conflicts What's interesting is that either changing the name of the table or the name of the field and these statements work. There is something about this particular combo that is problematic. sql>CREATE TABLE "sc_cat_item_delivery_plan1" ("script" string); operation successful sql>CREATE TABLE "sc_cat_item_delivery_plan" ("script1" string); operation successful I have a total of 2251 tables... all were created except for this one. I've tried to destroy and recreate the schema with no luck. Logs don't seem to indicate any issue and trying to use the debug on the client didn't get me anywhere. Thoughts? Suggestions? -- Please excuse typos... I can't type.

3 2

Issue while connecting to MonetDB via Perl cpan driver
by Sharma, Sreejith 15 Nov '18

15 Nov '18

Hi, We have a Perl code via which we are trying to connect to one of the MonetDB instance. Here we are using DBD::monetdb Perl cpan driver for connectivity. when we run the snippet, it just tries to connect to DB and seems hung, looks like it's awaiting for user input for password. Also, on the server side logs we see an error (challenge error) can connection will not be established. Here is a sample snippet which isn't working. Could anyone help with an idea or code snippet which works with Perl DBI for MonetDB connectivity? [cid:image001.png@01D47D35.C23AD040] #!/usr/bin/perl # use strict; use warnings; use DBI(); print "Attempting to connect...\n"; my $dbh = DBI->connect('dbi:monetdb:host=XXXX;port=55555', 'USERNAME', 'Password', { RaiseError => 1 } ); print "Connected. Trying SQL...\n"; my $sth = $dbh->prepare('SELECT * FROM env'); $sth->execute; $sth->dump_results; print "Done\n"; exit 0; Regards, Sreejith

1 0

Error: cannot bind delete column
by Doherty, Adam 15 Nov '18

15 Nov '18

Hello all: I have run across a show-stopping error, "cannot bind delete column" which renders Monet unusable until I restart the service. A batch import of events runs every 10 minutes and has been unable to complete for the last 28 hours because the service is not accepting commands. I'm not sure where to look for the source of this error. Restarting the service correct the issue, but this is a problem as the Monet instance drives live Tableau dashboards using this imported data. Where do I need to start looking to track down the source of this error? I am currently running 11.29.3 on CentOS 7. Adam

2 1

Adding columns to tables via binary bulk loading
by Anderson, David B 12 Nov '18

12 Nov '18

All, Is it possible to add a column to an existing table via binary bulk loading? We have a table with ~1 billion rows and we would like to add columns to the table, but would prefer bulk loads for speed. Also, does anyone have a snippet of code to create the binary file format that bulk loading expects. I've tried a couple versions (simple int and double), but the server wouldn't recognize the files. Thanks, Dave

2 1

Unable to get sys.queue for all the users
by Murthy, Gautham 07 Nov '18

07 Nov '18

Hello, We are unable to get the information for all active users sessions when we use sys.queue as monetdb user. Also, the tried as user other than monetdb, it returned no access, We had provided the user select privilege on the sys.queue view. [monetadmin@lnx1487 scripts]$ /monet_binaries/MonetDB-11.27.13_PY/bin/mclient mapi:monetdb://lnx1487:50053/DB_BWS_S Welcome to mclient, the MonetDB/SQL interactive terminal (Jul2017-SP4) Database: MonetDB v11.27.13 (Jul2017-SP4), 'mapi:monetdb://lnx1487:50053/DB_BWS_S' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select "user","active",count(*) as "No. Of Connections" from sys.sessions() group by "user","active"; +-------------------+--------+--------------------+ | user | active | No. Of Connections | +===================+========+====================+ | DM_POS_BWS_STG_P6 | true | 1 | | DM_POS_BWS_STG_P6 | false | 3 | | monetdb | true | 1 | +-------------------+--------+--------------------+ 3 tuples (6.027ms) sql>select * from sys.queue; +------+---------+----------------------------+----------------------------+----------+---------+--------+--------------------------+ | qtag | user | started | estimate | progress | status | tag | query | +======+=========+============================+============================+==========+=========+========+==========================+ | 3630 | monetdb | 2018-11-07 17:07:34.000000 | null | null | running | 3630@0 | select * from sys.queue; | +------+---------+----------------------------+----------------------------+----------+---------+--------+--------------------------+ 1 tuple (1.610ms) sql>\q [monetadmin@lnx1487 scripts]$ Thank You, Gautham

4 3

Cannot setup monetdb on Centos 7
by GAUQUELIN, Philippe 05 Nov '18

05 Nov '18

Hi Need help on setup. Thanks for your answer Trying to install I got this message. The GPG keys listed for the "MonetDB 7 - x86_64" repository are already installed but they are not correct for this package. More details below: []# yum reinstall https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm Loaded plugins: fastestmirror, langpacks MonetDB-release-epel.noarch.rpm | 6.4 kB 00:00:00 Examining /var/tmp/yum-root-x4i3eb/MonetDB-release-epel.noarch.rpm: MonetDB-release-epel-1.3-1.monetdb.noarch Resolving Dependencies --> Running transaction check ---> Package MonetDB-release-epel.noarch 0:1.3-1.monetdb will be reinstalled --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================= Package Arch Version Repository Size ======================================================================================================================================================================= Reinstalling: MonetDB-release-epel noarch 1.3-1.monetdb /MonetDB-release-epel.noarch 4.0 k Transaction Summary ======================================================================================================================================================================= Reinstall 1 Package Total size: 4.0 k Installed size: 4.0 k Is this ok [y/d/N]: y Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : MonetDB-release-epel-1.3-1.monetdb.noarch 1/1 Verifying : MonetDB-release-epel-1.3-1.monetdb.noarch 1/1 Installed: MonetDB-release-epel.noarch 0:1.3-1.monetdb Complete! [root@franalytic02 tmp]# []# rpm --import https://www.monetdb.org/downloads/MonetDB-GPG-KEY []# yum install MonetDB-SQL-server5 MonetDB-client Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: centos.crazyfrogs.org * epel: mirrors.ircam.fr * extras: centos.crazyfrogs.org * updates: centos.mirror.fr.planethoster.net Resolving Dependencies --> Running transaction check ---> Package MonetDB-SQL-server5.x86_64 0:11.31.11-20181011.el7 will be installed --> Processing Dependency: MonetDB5-server(x86-64) = 11.31.11-20181011.el7 for package: MonetDB-SQL-server5-11.31.11-20181011.el7.x86_64 --> Processing Dependency: libstream.so.11()(64bit) for package: MonetDB-SQL-server5-11.31.11-20181011.el7.x86_64 --> Processing Dependency: libmonetdb5.so.25()(64bit) for package: MonetDB-SQL-server5-11.31.11-20181011.el7.x86_64 --> Processing Dependency: libbat.so.17()(64bit) for package: MonetDB-SQL-server5-11.31.11-20181011.el7.x86_64 ---> Package MonetDB-client.x86_64 0:11.31.11-20181011.el7 will be installed --> Running transaction check ---> Package MonetDB.x86_64 0:11.31.11-20181011.el7 will be installed ---> Package MonetDB-stream.x86_64 0:11.31.11-20181011.el7 will be installed ---> Package MonetDB5-server.x86_64 0:11.31.11-20181011.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================= Package Arch Version Repository Size ======================================================================================================================================================================= Installing: MonetDB-SQL-server5 x86_64 11.31.11-20181011.el7 monetdb 854 k MonetDB-client x86_64 11.31.11-20181011.el7 monetdb 275 k Installing for dependencies: MonetDB x86_64 11.31.11-20181011.el7 monetdb 1.0 M MonetDB-stream x86_64 11.31.11-20181011.el7 monetdb 197 k MonetDB5-server x86_64 11.31.11-20181011.el7 monetdb 853 k Transaction Summary ======================================================================================================================================================================= Install 2 Packages (+3 Dependent packages) Total size: 3.1 M Installed size: 9.1 M Is this ok [y/d/N]: y Downloading packages: warning: /var/cache/yum/x86_64/7/monetdb/packages/MonetDB-stream-11.31.11-20181011.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID df0e54f3: NOKEY Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-MonetDB Importing GPG key 0x0583366F: Userid : "MonetDB Database System Packager <monet(a)cwi.nl>" Fingerprint: 213e 64dc d5dd c2c0 63cc 39fd 053c 3ed4 0583 366f Package : MonetDB-release-epel-1.3-1.monetdb.noarch (installed) >From : /etc/pki/rpm-gpg/RPM-GPG-KEY-MonetDB Is this ok [y/N]: y Importing GPG key 0x0583366F: Userid : "MonetDB Database System Packager <monet(a)cwi.nl>" Fingerprint: 213e 64dc d5dd c2c0 63cc 39fd 053c 3ed4 0583 366f Package : MonetDB-release-epel-1.3-1.monetdb.noarch (installed) >From : /etc/pki/rpm-gpg/RPM-GPG-KEY-MonetDB Is this ok [y/N]: y Public key for MonetDB-stream-11.31.11-20181011.el7.x86_64.rpm is not installed Failing package is: MonetDB-stream-11.31.11-20181011.el7.x86_64 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-MonetDB Philippe

2 1

Using capi.eval from MAL
by Renato Marroquín Mogrovejo 04 Nov '18

04 Nov '18

Hi all, I am trying to use the following patten from MAL[1] pattern eval(fptr:ptr,cpp:bit,expr:str,arg:any...):any... However, I am not sure how I should create the first parameter <fptr:ptr> from MAL. I am just trying to be able to call a C-UDF from MAL. From capi.c, it would seem that it is the client's context but I don't know how to get a handle of that. Any pointers are highly appreciated. Thanks! Renato M. [1] https://github.com/MonetDB/MonetDB/blob/2310e49a349ebcb8ff5b5c585928c8f4bd1…

2 4

Failed to import table 'tier3_score_201807_part53', Failed to extend the BAT, perhaps disk full
by Murthy, Gautham 04 Nov '18

04 Nov '18

Hello, We are hitting the below error while loading data into table, There is no space crunch or memory (resource crunch). Please let us know if there are any restrictions in loading data. sql>COPY INTO "DM_POS_PROSCORES_MRT_P1".tier3_score_201807_part53 FROM '/backup8/schema_backups/tier3_score_201807_part53'; Failed to import table 'tier3_score_201807_part53', Failed to extend the BAT, perhaps disk full [monetadmin@lnx1538 monet_data01]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdbd version MonetDB Database Server v1.7 (Jul2017-SP4) You have mail in /var/spool/mail/monetadmin [monetadmin@lnx1538 monet_data01]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdb version MonetDB Database Server Toolkit v1.1 (Jul2017-SP4) [monetadmin@lnx1538 monet_data01]$ mclient -version mclient, the MonetDB interactive terminal (Jul2017-SP4) support for command-line editing compiled-in character encoding: UTF-8 [monetadmin@lnx1538 monet_data01]$ df -hP /monet_data01 Filesystem Size Used Avail Use% Mounted on /dev/mapper/encrmonet_data01lv 12T 7.0T 4.4T 62% /monet_data01 [monetadmin@lnx1538 monet_data01]$ ps -ef|grep -i monetdbd monetad+ 331928 1 0 Nov02 ? 00:00:14 /monet_binaries/MonetDB-11.27.13_PY/bin/monetdbd start /monet_data01/DBFARM_TEST01 monetad+ 333365 332892 0 10:57 pts/9 00:00:00 grep --color=auto -i monetdbd monetad+ 405353 1 0 Oct31 ? 00:00:30 /monet_binaries/MonetDB-11.27.13_PY/bin/monetdbd start /monet_data01/DBFARM_PROSCORE_P [monetadmin@lnx1538 monet_data01]$ [monetadmin@lnx1538 monet_data01]$ free -g total used free shared buff/cache available Mem: 1007 18 302 7 685 978 Swap: 31 0 31 [monetadmin@lnx1538 monet_data01]$ Thank You, Gautham

3 3

Append an array of values to a BUN
by Anton Kravchenko 02 Nov '18

02 Nov '18

Hi there, Is there a way to append an array of values to a BUN, instead one value at a time? sql_table *tbl = mvc_bind_table(m, sch, "atable"); sql_column *col = mvc_bind_column(m, tbl, "acolumn"); BAT *tmp1 = COLnew(0, TYPE_lng, (BUN)nrows, TRANSIENT); long long v_long[3] = {100000000009,100000000010,100000000011}; for (irow=0; irow<nrows; irow++) BUNappend(tmp1, &v_long[irow], FALSE); mvc_append_column(m->session->tr, col1, tmp1); mvc_commit(m, 0, tr->name); bat_destroy(tmp1); Thanks, Anton

2 5

Circular Memory Only Database
by Doherty, Adam 26 Oct '18

26 Oct '18

Hello all: I am wondering if anyone has a remote table strategy such that "hot data" is stored in a table that is backed by RAM. I have a table with approximately 1 billion events that grows by approximately 1,000 events a minute. Table is a merge table of merge tables, the parent merge table has a child merge table for every year of data under those child tables are source tables that make up data for areas. So Area1_2017, Area2_2017 -> Events_2017 Area1_2018, Area2_2018 -> Events_2018 Events_2017, Events_2018 -> Events I am finding that as I add more data the system seems to slow down. I am wanting to keep a cyclical table of say the last 6 months from the Events table in a table backed solely by RAM So a query to Recent_Events would remain very fast or at the very least constant. This should be achievable with tmpfs. I am currently running 11.29.3 on CentOS 7 on VMware 6.7 Server has 128GB of RAM and 16 cores. Adam

2 1

SQLSTATE Error not showing up in mclient Aug2018 version (Aug2018 (11.31.7))
by Sharma, Sreejith 22 Oct '18

22 Oct '18

All, Do we have a way to compile mclient along with server source code? We are not able to see SQLSTATE error code on concurrency with Aug2018 (11.31.7) server version along with mclient Aug2018. Would like to see if we need to specifically compile client too? ERROR = !COMMIT: transaction is aborted because of concurrency conflicts, will ROLLBACK instead CODE: 40000 Regards, Sreejith

2 3

Re: users-list Digest, Vol 74, Issue 3
by Oxana Dolmatova 17 Oct '18

17 Oct '18

Hi Stefan, Thank you for the information, now I see the point. >Did you by any chance try any other DBMSs with your "non-standard" workload? >If so, how do those experiences and limitations compare to MonetDB? I did not run tests over conventional DBMSs because they have stricter limits compared to MonetDB (e.g. Postgres is limited to 1600 columns), but only over array DBMSs or math packages. Obviously, their column scalability is better, but they can not handle RA that efficiently. Best regards, Oksana On Tue, 16 Oct 2018 at 12:16, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. Re: Column limit (Oxana Dolmatova) > 2. Re: Column limit (Stefan Manegold) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 15 Oct 2018 17:14:14 +0200 > From: Oxana Dolmatova <oxana.dolmatova(a)gmail.com> > To: E.Rozenberg(a)cwi.nl > Cc: users-list(a)monetdb.org > Subject: Re: Column limit > Message-ID: > <CAJJ2wAQi+4S+= > e9BfbStuJxD7izNh8z-mcBF8FYKxagjj5oRkA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Eyal, > > Sorry for the late reply. Yes, there is no problem with large number of > columns in sparse representation of arrays (i.e., dimension, dimension, > value). > However, in order to perform relational operations, this structure should > be pivoted first. When matrices are represented by tables, both algebras > can be applied to the same object, > so I'm specifically looking for the main bottleneck of processing hundreds > of thousands of columns in MonetDB. > > Best regards, > Oksana > > > On Wed, 10 Oct 2018 at 13:40, Eyal Rozenberg <E.Rozenberg(a)cwi.nl> wrote: > > > If you want to model an NxM matrix of elements of type MYTYPE in a DBMS > > (not just MonetDB), you would typically have a table with 3 columns: > > > > CREATE TABLE my_matrix ( > > "row" INTEGER NOT NULL, > > "col" INTEGER NOT NULL, > > "cell_value" MYTYPE NOT NULL, > > PRIMARY KEY ("row", "col") > > ); > > > > of length N x M. > > > > Anyways, I'm not the expert on Arrays in our group so I'll let others > > chime in. > > > > Eyal > > > > > > On 10/10/2018 13:01, Oxana Dolmatova wrote: > > > Hello, > > > > > > Thank you for all the replies and information! > > > The version is 11.23, plus I'm using the cloud platform > > with 4cpu-16ram-hpc. > > > The number of rows was just 1K for 15K attributes in a table. > > > I understand that MonetDB was made with no intention to create and > > > process tables with that many columns. But I'm interested in matrix > > > operations integration, so I would like to know what is the main > > > bottleneck to process such data (matrix-like sized) in the system. > > > > > > Best regards, > > > Oksana > > > > > > > > > On Tue, 9 Oct 2018 at 21:02, Eyal Rozenberg <eyalroz(a)technion.ac.il > > > <mailto:eyalroz@technion.ac.il>> wrote: > > > > > > I'll add two observations to what Zhang Ying said. > > > > > > 1. Internally in MonetDB, the index of a column (or a BAT) in > > MonetDB's > > > BBP (BAT Buffer Pool) is a signed 32-bit integer (on most > > platforms), so > > > there isn't a theoretical problem having more than 15,000 columns; > > but > > > MonetDB was not designed with that many tables in mind. The "long > > > dimension" is expected to be the length of columns. > > > > > > 2. When you have as many as 15,000 columns, you're not simply > finding > > > more attributes of the same items. So, for example, a person has a > > first > > > name, last name, year of birth, etc. etc. - but after several > dozens > > of > > > these you're probably adding columns such as "data relevant to this > > > person at point X", "data relevant to this person at point Y" and > so > > on. > > > > > > In your case, I'm guessing you have a lot of points and hence a lot > > of > > > columns with data for your different items. What you should > consider > > > doing instead is having an extra "coordinate" column. Thus, in the > > > example I gave, I would drop most of the columns, in favor of a > > second > > > table named "person_data". This table's columns would be > "person_id" > > (or > > > some other key columns), "sample_point" and "value". Thus instead > of > > M > > > columns and N records you would have N x M records with 3 columns. > > > > > > Eyal > > > > > > > > > > > > On 09/10/2018 16:50, Ying Zhang wrote: > > > > Hai Oksana, > > > > > > > > A.f.a.I.k., there is no explicit limit on the #columns you can > put > > > in a relation (did you mean ?table??). The reason that mserver5 > > > stopped responding is probably because it is busy with I/O. > > > > > > > > A long time ago, I once tried to create 1M tables, each with 3 > > > columns. After about a day or something, MonetDB has created ~400K > > > tables, and I could hear my HDD being constantly busy. MonetDB was > > > still working, but that was already a sufficient evidence that that > > > idea wasn?t going to work. > > > > > > > > So the question is actually what do you want to achieve? > > > > > > > > Regards, > > > > Jennie > > > > > > > > > > > >> On 9 Oct 2018, at 13:43, Oxana Dolmatova > > > <oxana.dolmatova(a)gmail.com <mailto:oxana.dolmatova@gmail.com>> > > wrote: > > > >> > > > >> Hello, > > > >> > > > >> Could you tell the limitation for number of columns in a > relation > > > and the main reason for that limitation. So far I could create > > > relations with up to 15000 columns. When the number of columns is > > > bigger mserver5 stops responding. > > > >> > > > >> Thank you and best regards, > > > >> Oksana Dolmatova > > > >> _______________________________________________ > > > >> users-list mailing list > > > >> users-list(a)monetdb.org <mailto:users-list@monetdb.org> > > > >> https://www.monetdb.org/mailman/listinfo/users-list > > > > > > > > _______________________________________________ > > > > users-list mailing list > > > > users-list(a)monetdb.org <mailto:users-list@monetdb.org> > > > > https://www.monetdb.org/mailman/listinfo/users-list > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Oksana Dolmatova > > > > > > _______________________________________________ > > > users-list mailing list > > > users-list(a)monetdb.org > > > https://www.monetdb.org/mailman/listinfo/users-list > > > > > > > > -- > Best regards, > Oksana Dolmatova >

1 0

Announcement: New Aug2018-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 17 Oct '18

17 Oct '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2018-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Aug 2018-SP1 bugfix release (11.31.11) Build Environment * On Ubuntu 18.10 (Cosmic Cuttlefish), the libmonetdb5-server-bam package cannot be built because of an incompatibility in the libbam library (it cannot be used in a shared object. Whole System * Some subtle dependencies between RPMs have been fixed. Bug Fixes * 6640: timestamp_to_str returning incorrectly adjusted results * 6641: race condition in SQL UDF with update * 6642: Hanging query * 6648: key property potentially wrong after type conversion * 6649: Projection inside within transaction gives wrong results

1 0

Column limit
by Oxana Dolmatova 15 Oct '18

15 Oct '18

Hello, Could you tell the limitation for number of columns in a relation and the main reason for that limitation. So far I could create relations with up to 15000 columns. When the number of columns is bigger mserver5 stops responding. Thank you and best regards, Oksana Dolmatova

6 7

monetdb discover not finding other db's
by Frik Lubbe 24 Sep '18

24 Sep '18

Hi all, I relatively new to MonetDB, but have many year exposure to database enignes. I have 2 x vm's, almost identical with the exception of IP's and host names. both are Ubuntu Server 18.04 LTS. I have installed MonetDB on both VM's and created a db farm and a database on each. When running 'monetdb discover' on any of the VM's it finds nothing. Any help is welcome please. Regards, Frik Outputs from 'monetdbd get all /data/monetdb/dbfarm/' *VM 1:* root@ubuntu-srv-001:~# monetdbd start /data/monetdb/dbfarm/ another monetdbd is already running root@ubuntu-srv-001:~# monetdbd get all /data/monetdb/dbfarm/ property value hostname ubuntu-srv-001 dbfarm /data/monetdb/dbfarm/ status monetdbd[1941] 1.7 (Aug2018) is serving this dbfarm mserver /usr/bin/mserver5 logfile /data/monetdb/dbfarm//merovingian.log pidfile /data/monetdb/dbfarm//merovingian.pid sockdir /tmp listenaddr 0.0.0.0 port 50000 exittimeout 60 forward proxy discovery yes discoveryttl 600 control yes passphrase {SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35 mapisock /tmp/.s.monetdb.50000 controlsock /tmp/.s.merovingian.50000 root@ubuntu-srv-001:~# monetdb discover root@ubuntu-srv-001:~# *VM 2:* root@ubuntu-srv-002:~# monetdbd get all /data/monetdb/dbfarm/ property value hostname ubuntu-srv-002 dbfarm /data/monetdb/dbfarm/ status monetdbd[2373] 1.7 (Aug2018) is serving this dbfarm mserver /usr/bin/mserver5 logfile /data/monetdb/dbfarm//merovingian.log pidfile /data/monetdb/dbfarm//merovingian.pid sockdir /tmp listenaddr 0.0.0.0 port 60001 exittimeout 60 forward proxy discovery yes discoveryttl 600 control yes passphrase {SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35 mapisock /tmp/.s.monetdb.60001 controlsock /tmp/.s.merovingian.60001 root@ubuntu-srv-002:~#

1 0

SQLcopyinto.java
by Roberto Cornacchia 20 Sep '18

20 Sep '18

I'm trying to debug n issue I have with data disappearing after a database restart. Because part of the process for creating the table involves a mixed Mapi / JDBC approach as described in monetdb-java/example/SQLcopyinto.java, I started from this example. When I run it though, I get a final count of 0 (should be 100). Can anyone confirm? Mar2018, JDBC 2.28 Roberto

2 1

Join Order Benchmark on MonetDB
by Junxiong Wang 19 Sep '18

19 Sep '18

Hello, I am currently evaluating the performance of MonetDB on the Join Order Benchmark, proposed by Leis et al.(http://www.vldb.org/pvldb/vol9/p204-leis.pdf <http://www.vldb.org/pvldb/vol9/p204-leis.pdf>). There are a few queries where performance seems suboptmal, e.g., query 16b and 17e (https://github.com/gregrahn/join-order-benchmark <https://github.com/gregrahn/join-order-benchmark>): MonetDB takes around 400 seconds to evaluate query 17e with a single thread, more than 20 seconds with 48 threads, Postgres takes only around 9 seconds. I suspect that the query optimizer selects the wrong plan, this is the output of the plan command. | project ( | | | group by ( | | | | join ( | | | | | join ( | | | | | | join ( | | | | | | | join ( | | | | | | | | join ( | | | | | | | | | join ( | | | | | | | | | | table(sys.movie_companies) [ "movie_companies"."movie_id" NOT NULL as "mc"."movie_id", "movie_companies"."company_id" NOT NULL as "mc"."company_id" ] COUNT , | | | | | | | | | | select ( | | | | | | | | | | | table(sys.company_name) [ "company_name"."id" NOT NULL HASHCOL as "cn"."id", "company_name"."country_code" as "cn"."country_code" ] COUNT | | | | | | | | | | ) [ "cn"."country_code" = varchar(255) "[us]" ] | | | | | | | | | ) [ "mc"."company_id" NOT NULL = "cn"."id" NOT NULL HASHCOL ], | | | | | | | | | table(sys.title) [ "title"."id" NOT NULL HASHCOL as "t"."id" ] COUNT | | | | | | | | ) [ "t"."id" NOT NULL HASHCOL = "mc"."movie_id" NOT NULL ], | | | | | | | | table(sys.cast_info) [ "cast_info"."person_id" NOT NULL as "ci"."person_id", "cast_info"."movie_id" NOT NULL as "ci"."movie_id" ] COUNT | | | | | | | ) [ "ci"."movie_id" NOT NULL = "t"."id" NOT NULL HASHCOL , "ci"."movie_id" NOT NULL = "mc"."movie_id" NOT NULL ], | | | | | | | table(sys.name) [ "name"."id" NOT NULL HASHCOL as "n"."id", "name"."name" NOT NULL as "n"."name" ] COUNT | | | | | | ) [ "n"."id" NOT NULL HASHCOL = "ci"."person_id" NOT NULL ], | | | | | | table(sys.movie_keyword) [ "movie_keyword"."movie_id" NOT NULL as "mk"."movie_id", "movie_keyword"."keyword_id" NOT NULL as "mk"."keyword_id" ] COUNT | | | | | ) [ "t"."id" NOT NULL HASHCOL = "mk"."movie_id" NOT NULL, "ci"."movie_id" NOT NULL = "mk"."movie_id" NOT NULL, "mc"."movie_id" NOT NULL = "mk"."movie_id" NOT NULL ], | | | | | select ( | | | | | | table(sys.keyword) [ "keyword"."id" NOT NULL HASHCOL as "k"."id", "keyword"."keyword" NOT NULL as "k"."keyword" ] COUNT | | | | | ) [ "k"."keyword" NOT NULL = clob "character-name-in-title" ] | | | | ) [ "mk"."keyword_id" NOT NULL = "k"."id" NOT NULL HASHCOL ] | | | ) [ ] [ sys.min no nil ("n"."name" NOT NULL) NOT NULL as "L3"."L3" ] | | ) [ "L3"."L3" NOT NULL as "L4"."member_in_charnamed_movie" ] | +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I was wondering whether anyone might have a hint about what the problem could be and how one could improve performance for those specific queries? Thanks a lot!

1 0

Measure the memory footprint of Monetdb from an SQL client
by Mahmoud Mohsen 12 Sep '18

12 Sep '18

Hello, Is there away to measure the physical or virtual memory consumption of Monetdb using an SQL query. Something similar to select * from storage() where table_name = name; I believe storage() helps with the disk consumption not main memory consumption, am I right? Thank you a lot in advance, Mahmoud Mohsen

2 1

COPY INTO FILE - Folder permission
by Ariel Abadi 10 Sep '18

10 Sep '18

Hi! We are running some "exports" using the COPY INTO FILE, the problem we are facing is that the file created by monetdb is Read Write ONLY by the user running the database. -rw------- 1 root monet 9477 Sep 7 15:08 exp.csv Is there any way so the created file has also RW permission to the group and NOT only the user. In this case, we are using an NFS share so we are not able to use setfacl. Does anybody has an idea how to solve this ? Thanks a lot in advance! Rgds Ariel

5 8

About Monetdb Query Plan
by Junxiong Wang 07 Sep '18

07 Sep '18

Hello, I am currently working on a project related to query optimization I have several questions, 1) Is it possible to force the query engine to use the specified join order? 2) When I change the table order in the “FROM” clause, the query time varies a lot (many orders of magnitude)? Does the table order in the “FROM” clause affect the query plan? 3) I want to know the details of MonetDB query plan. When I use “explain” command, I just get the MAL code. Are there any tools to visualize it, just like PostgreSQL “analyze" command? Thanks a lot.

2 1

About Monetdb Query Plan
by Junxiong Wang 06 Sep '18

06 Sep '18

Hello, I am currently working on a project related to query optimization I have several questions, 1) Is it possible to force the query engine to use the specified join order? 2) When I change the table order in the “FROM” clause, the query time varies a lot (many orders of magnitude)? Does the table order in the “FROM” clause affect the query plan? 3) I want to know the details of MonetDB query plan? But when I use “explain” command, I just get the MAL code. Are there any tools to visualize it, just like PostgreSQL “analyze" command? Thanks a lot.

1 0

How to enable SQLSTATE error code in mclient
by Murthy, Gautham 06 Sep '18

06 Sep '18

Hello, How do we enable SQLSTATE error code on mclient ? Thank You, Gautham

1 0