- users-list - monetdb.org

New client library for PHP
by Tamas Bolner 10 Apr '20

10 Apr '20

Hi, I spent the past nights creating a client library for PHP, because I found too many problems with the official one. Just for the heads up: https://github.com/bolner/MonetDB-PHP-Deux Happy Easter! Tamas

1 0

RE: System Catalog - Objects creation date and time.
by Suresh Devarakonda 08 Apr '20

08 Apr '20

Hi, I just wanted to see when the objects such as tables created in MonetDB. I have looked at below note, but couldn't find the catalog tables that provides the table creation time so wondering if MonetDB has that capability. Can someone let me know at the earliest. Your early response is very much appreciated. https://www.monetdb.org/Documentation/SystemCatalog [cid:image002.jpg@01D60C19.28554180] Suresh Devarakonda 84.51<http://www.8451.com/>° TDO - Database Support Team D: 513.632.3692 M: 859.512.1839 A: 100 W. 5th Street, Cincinnati, Ohio 45202 Cincinnati | Chicago | New York | Portland Facebook<https://www.facebook.com/8451group/?ref=aymt_homepage_panel> LinkedIn<https://www.linkedin.com/company/84-51/?trk=biz-companies-cym> Instagram<https://www.instagram.com/84.51/> Twitter<https://twitter.com/8451group> Please consider the environment before printing this email.

2 1

Running two databases with forward set to redirect (monetdbd)
by stephenb 08 Apr '20

08 Apr '20

We have an environment that does not support Unix domain sockets well. We thus configured monetdbd with forward=redirect, as we note from the monetdbd man page that "for local databases, the proxy method uses a UNIX domain socket feature" We can create two database, but trying to connect (on the default 50000 port) to them only the first-started database is available. If we try to connect to the second we get the error "request for database 'dev_sb2', but this is database 'dev_sb', did you mean to connect to monetdbd instead?". With some experimentation we have found that starting the databases directly using mserver5 and specifying different ports (--set mapi_port=50001 etc) we can connect to both databases, but this does complicate things and we would prefer to use the monetdb command for starting, stopping, creating etc. How can we configure monetdbd (or perhaps individual databases?) so that we can connect to two local databases through monetdbd and manage them using monetdb? Thanks Steve

2 2

Fwd: [Help] create a new db user but need to add prefix 'sys.' in order to execute timestamp_to_str() function
by Kevin Chen 07 Apr '20

07 Apr '20

Hi, I have a new db user created by following scripts: CREATE USER "imdb" WITH PASSWORD 'imdb' NAME 'IMDB Explorer' SCHEMA "sys"; CREATE SCHEMA "imdb" AUTHORIZATION "imdb"; ALTER USER "imdb" SET SCHEMA "imdb"; When I tried to execute the function "timestamp_to_str", I have to add "sys." prefix to make it effective. "date_trunc" also needed to do so. The default user "monetdb" can run the function directly without the prefix. What should I do to perform the same way as user "monetdb" ? sql>select timestamp_to_str(localtimestamp, '%Y-%m-%d %H:%M:%S %z'); SELECT: no such binary operator 'timestamp_to_str(timestamp,char)' sql>select sys.timestamp_to_str(localtimestamp, '%Y-%m-%d %H:%M:%S %z'); +---------------------------+ | L2 | +===========================+ | 2020-04-06 20:37:39 +0800 | +---------------------------+ 1 tuple sql>select date_trunc('minute', timestamp '2119-02-17 02:08:12.345678'); SELECT: no such binary operator 'date_trunc(char,timestamp)' sql>select sys.date_trunc('minute', timestamp '2119-02-17 02:08:12.345678'); +----------------------------+ | L2 | +============================+ | 2119-02-17 02:08:00.000000 | +----------------------------+ 1 tuple

2 1

Fwd: BULK UPLOAD very poor performance
by Ariel Abadi 01 Apr '20

01 Apr '20

Hi! After a big battle against the DB, I found 2 issues. 1) Running the COPY RECORDS FROM *STDIN *and from *FILE*. Is totally different in terms of performance. While the first one last 24 minutes, the second one last only *90 secs.* 2) As I had generated all those files thru msqldump (which it states the "CREATE TABLE" instruction and then the COPY RECORDS)... I try ro run the COPY *XXXX OFFSET *RECORDS (in order to avoid the create table statement)... and the query, because of the word "STDIN" in the file. It was not OFFSETTING all the rows I was asking for. I hope I've been clear... if not please let me know! Ariel On Tue, Mar 24, 2020 at 11:33 PM Martin Kersten <martin.kersten(a)cwi.nl> wrote: > Hi > > I am not sure what issue you are now dealing with, but since you are a > company > that seems to use (or considering) MonetDB in your commercial offerings, > I gladly refer you to MonetDBSolutions(*), where we can give you the > commercial > support needed to diagnose your issues. > > regards Martin > > (*) https://www.monetdbsolutions.com/ > > On 24/03/2020 23:55, Ariel Abadi wrote: > > Thanks Martin for your prompt response > > But I did and msqldump from other database, and now the upload is being > done thru the BULK LOAD. > > > > /COPY 14173375 RECORDS INTO "sb_traf"."nrm_20180818" FROM stdin USING > DELIMITERS '\t','\n','"';/ > > > > > > Any other idea ? :'( > > > > > > > > > > On Tue, Mar 24, 2020 at 7:37 PM Martin Kersten <martin.kersten(a)cwi.nl > <mailto:martin.kersten@cwi.nl>> wrote: > > > > Hi, > > > > given the numbers I suspect you are loading them as SQL insert > statements under autocommit mode, > > Please read the documentation on Bulk loading. > > > > regards,Martin > > Sent from my iPad > > > >> On 24 Mar 2020, at 23:12, Ariel Abadi <aabadi(a)starbi.com <mailto: > aabadi(a)starbi.com>> wrote: > >> > >> > >> Hi! > >> Maybe somebody can help me, please! > >> > >> I'm trying to upload into Monet a file with 14MM records. Is taking > more than *24 minutes.* > >> It is a fresh new database. > >> > >> Enclosed you will find the server configuration. > >> > >> Monetdb Version Nov2019-SP3 > >> > >> > >> *[root@mn tmp]$ lscpu* > >> Architecture: x86_64 > >> CPU op-mode(s): 32-bit, 64-bit > >> Byte Order: Little Endian > >> CPU(s): 40 > >> On-line CPU(s) list: 0-39 > >> Thread(s) per core: 2 > >> Core(s) per socket: 10 > >> Socket(s): 2 > >> NUMA node(s): 2 > >> Vendor ID: GenuineIntel > >> CPU family: 6 > >> Model: 85 > >> Model name: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz > >> Stepping: 4 > >> CPU MHz: 800.479 > >> BogoMIPS: 4400.00 > >> Virtualization: VT-x > >> L1d cache: 32K > >> L1i cache: 32K > >> L2 cache: 1024K > >> L3 cache: 14080K > >> NUMA node0 CPU(s): > 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 > >> NUMA node1 CPU(s): > 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39 > >> Flags: fpu vme de pse tsc msr pae mce cx8 apic sep > mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts > rep_good nopl xtopology nonstop_tsc > >> cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est > tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe > popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm > 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 > >> invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow > vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep > bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap > clflushopt clwb intel_pt avx512cd avx512bw > >> avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc > cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear > flush_l1d > >> > >> > >> *[root@mn tmp]$ cat /proc/meminfo* > >> > >> MemTotal: 83886080 kB > >> MemFree: 10535812 kB > >> MemAvailable: 80159744 kB > >> Buffers: 0 kB > >> Cached: 69623932 kB > >> SwapCached: 0 kB > >> Active: 6625436 kB > >> Inactive: 63737872 kB > >> Active(anon): 324456 kB > >> Inactive(anon): 417184 kB > >> Active(file): 6300980 kB > >> Inactive(file): 63320688 kB > >> Unevictable: 0 kB > >> Mlocked: 5320 kB > >> SwapTotal: 8388604 kB > >> SwapFree: 8381988 kB > >> Dirty: 220 kB > >> Writeback: 0 kB > >> AnonPages: 13626052 kB > >> Mapped: 186368 kB > >> Shmem: 1800 kB > >> KReclaimable: 4134960 kB > >> Slab: 0 kB > >> SReclaimable: 0 kB > >> SUnreclaim: 0 kB > >> KernelStack: 16720 kB > >> PageTables: 56500 kB > >> NFS_Unstable: 0 kB > >> Bounce: 0 kB > >> WritebackTmp: 0 kB > >> CommitLimit: 74123472 kB > >> Committed_AS: 18337812 kB > >> VmallocTotal: 34359738367 kB > >> VmallocUsed: 415836 kB > >> VmallocChunk: 0 kB > >> Percpu: 39552 kB > >> HardwareCorrupted: 0 kB > >> AnonHugePages: 0 kB > >> ShmemHugePages: 0 kB > >> ShmemPmdMapped: 0 kB > >> CmaTotal: 0 kB > >> CmaFree: 0 kB > >> HugePages_Total: 0 > >> HugePages_Free: 0 > >> HugePages_Rsvd: 0 > >> HugePages_Surp: 0 > >> Hugepagesize: 2048 kB > >> Hugetlb: 0 kB > >> DirectMap4k: 816396 kB > >> DirectMap2M: 82513920 kB > >> DirectMap1G: 52428800 kB > >> > >> > >> > >> > >> _______________________________________________ > >> users-list mailing list > >> users-list(a)monetdb.org <mailto:users-list@monetdb.org> > >> https://www.monetdb.org/mailman/listinfo/users-list > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org <mailto:users-list@monetdb.org> > > https://www.monetdb.org/mailman/listinfo/users-list > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org > > https://www.monetdb.org/mailman/listinfo/users-list > > >

5 11

BULK UPLOAD very poor performance
by Ariel Abadi 24 Mar '20

24 Mar '20

Hi! Maybe somebody can help me, please! I'm trying to upload into Monet a file with 14MM records. Is taking more than *24 minutes.* It is a fresh new database. Enclosed you will find the server configuration. Monetdb Version Nov2019-SP3 *[root@mn tmp]$ lscpu* Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 40 On-line CPU(s) list: 0-39 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz Stepping: 4 CPU MHz: 800.479 BogoMIPS: 4400.00 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 1024K L3 cache: 14080K NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear flush_l1d *[root@mn tmp]$ cat /proc/meminfo* MemTotal: 83886080 kB MemFree: 10535812 kB MemAvailable: 80159744 kB Buffers: 0 kB Cached: 69623932 kB SwapCached: 0 kB Active: 6625436 kB Inactive: 63737872 kB Active(anon): 324456 kB Inactive(anon): 417184 kB Active(file): 6300980 kB Inactive(file): 63320688 kB Unevictable: 0 kB Mlocked: 5320 kB SwapTotal: 8388604 kB SwapFree: 8381988 kB Dirty: 220 kB Writeback: 0 kB AnonPages: 13626052 kB Mapped: 186368 kB Shmem: 1800 kB KReclaimable: 4134960 kB Slab: 0 kB SReclaimable: 0 kB SUnreclaim: 0 kB KernelStack: 16720 kB PageTables: 56500 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 74123472 kB Committed_AS: 18337812 kB VmallocTotal: 34359738367 kB VmallocUsed: 415836 kB VmallocChunk: 0 kB Percpu: 39552 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 816396 kB DirectMap2M: 82513920 kB DirectMap1G: 52428800 kB

2 2

Slowdown behaviour after loading large amounts of data
by stephenb 23 Mar '20

23 Mar '20

We believe we might be observing the behaviour described here: https://www.monetdb.org/blog/monetdb-sql-transaction-management-scheme Specifically “At certain intervals, if the database is idle, the store manager will merge the updates from the WAL into the primary storage” We perform some heavy updates to the database (we have experimented with putting these in a single transaction or having separate transactions for each, but the behaviour is the same). The updates are basically dropping tables, recreating them and then a COPY INTO. Once the final load has finished, and has been committed, for around 30 seconds we can query the database, but then for a period of several minutes it basically locks up. I wonder if there is any way to control this behaviour? Regards Steve

1 0

stethoscope tool
by Suresh Devarakonda 12 Mar '20

12 Mar '20

Thank you Sjoerd Mullender. I am able to install it. ====== Doing sudo yum install /usr/bin/stethoscope should do it. That will install the package MonetDB-client-tools. On 09/03/2020 16.11, Suresh Devarakonda wrote: > Hi, > > > > I would like to use the stethoscope tool to monitor the performance of > queries. But, I do not find the tool in the MonetDB software that I > installed on my server. Can you help me to get the stethoscope tool. > > > > [root at xxxxxx<https://www.monetdb.org/mailman/listinfo/users-list> ~]# yum list installed |grep -i monet > > MonetDB.x86_64 11.35.9-20191218.el7 @monetdb > > MonetDB-SQL-server5.x86_64 11.35.9-20191218.el7 @monetdb > > MonetDB-client.x86_64 11.35.9-20191218.el7 @monetdb > > MonetDB-client-odbc.x86_64 11.35.9-20191218.el7 @monetdb > > MonetDB-release-epel.noarch 1.3-1.monetdb installed > > MonetDB-stream.x86_64 11.35.9-20191218.el7 @monetdb > > MonetDB5-server.x86_64 11.35.9-20191218.el7 @monetdb > > > > > > > > *Suresh Devarakonda* > > *84.51* <http://www.8451.com/>*° * > > *TDO - Database Support Team* > > *D:*513.632.3692 > > *M*: 859.512.1839 > > *A: *100 W. 5^th Street, Cincinnati, Ohio 45202 > > > > Cincinnati *|* Chicago *|* New York *|* Portland > > Facebook > <https://www.facebook.com/8451group/?ref=aymt_homepage_panel> LinkedIn > <https://www.linkedin.com/company/84-51/?trk=biz-companies-cym> Instagram > <https://www.instagram.com/84.51/> Twitter > <https://twitter.com/8451group> > > Please consider the environment before printing this email. > > > > > > > _______________________________________________ > users-list mailing list > users-list at monetdb.org<https://www.monetdb.org/mailman/listinfo/users-list> > https://www.monetdb.org/mailman/listinfo/users-list > ________________________________ * Previous message (by thread): stethoscope tool<https://www.monetdb.org/pipermail/users-list/2020-March/010593.html> * Messages sorted by: [ date ]<https://www.monetdb.org/pipermail/users-list/2020-March/date.html#10594> [ thread ]<https://www.monetdb.org/pipermail/users-list/2020-March/thread.html#10594> [ subject ]<https://www.monetdb.org/pipermail/users-list/2020-March/subject.html#10594> [ author ]<https://www.monetdb.org/pipermail/users-list/2020-March/author.html#10594>

1 0

stethoscope tool
by Suresh Devarakonda 09 Mar '20

09 Mar '20

Hi, I would like to use the stethoscope tool to monitor the performance of queries. But, I do not find the tool in the MonetDB software that I installed on my server. Can you help me to get the stethoscope tool. [root@xxxxxx ~]# yum list installed |grep -i monet MonetDB.x86_64 11.35.9-20191218.el7 @monetdb MonetDB-SQL-server5.x86_64 11.35.9-20191218.el7 @monetdb MonetDB-client.x86_64 11.35.9-20191218.el7 @monetdb MonetDB-client-odbc.x86_64 11.35.9-20191218.el7 @monetdb MonetDB-release-epel.noarch 1.3-1.monetdb installed MonetDB-stream.x86_64 11.35.9-20191218.el7 @monetdb MonetDB5-server.x86_64 11.35.9-20191218.el7 @monetdb [cid:image001.jpg@01D5F602.57C58E70] Suresh Devarakonda 84.51<http://www.8451.com/>° TDO - Database Support Team D: 513.632.3692 M: 859.512.1839 A: 100 W. 5th Street, Cincinnati, Ohio 45202 Cincinnati | Chicago | New York | Portland Facebook<https://www.facebook.com/8451group/?ref=aymt_homepage_panel> LinkedIn<https://www.linkedin.com/company/84-51/?trk=biz-companies-cym> Instagram<https://www.instagram.com/84.51/> Twitter<https://twitter.com/8451group> Please consider the environment before printing this email.

2 1

SP2 R integration
by ALEX OSSIPOV 22 Feb '20

22 Feb '20

I am not able to get R part working in latest SP2 and getting this error: LoaderException:loadLibrary:Loading error failed to open lib rary rapi (from within file '/usr/lib64/monetdb5/lib_rapi.so'): libR.so: cannot open shared object file: No such file or directory I use Centos 7 package MonetDB-R.x86_64_11.35.15-20200217.el7 and wonder if it was build with --enable-R-shlib option. SP1 did not have this problem Thanks Alex

2 2

Announcement: New Nov2019-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Feb '20

21 Feb '20

The MonetDB team at MonetDB BV is pleased to announce the Nov2019-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://www.monetdb.org/downloads/>. Nov 2019-SP2 bugfix release (11.35.15) Bug Fixes * 6805: Using the cascade operator in a drop table statement ends in an exit from the Monetdb shell. * 6807: Median_avg and quantile_avg ignore NULL values * 6815: query with ifthenelse() crashes mserver5 * 6816: Monetdb Crashes on INSERT statement after ALTER statement in another connection * 6817: running analyze on a schema which contains a stream table stops with an error * 6819: functions do not persist

1 0

dropped functions
by ALEX OSSIPOV 18 Feb '20

18 Feb '20

I noticed every time Monetdb service is bounced all created functions disappear from database. Is it a bug or a way functions are handled in MonetDB? Thanks Alex.

2 1

search links are broken
by ALEX OSSIPOV 17 Feb '20

17 Feb '20

All links in Search results point to localhost:8080 instead of site. Can it be fixed? Thanks Alex

2 1

Include additional libraries
by Oxana Dolmatova 13 Feb '20

13 Feb '20

Hello, Could you tell what would be the easiest way to include additional libraries into the compilation process of MonetDB? More specifically I'm interested in calling MKL operations inside BAT Algebra operations. For example, mkl standalone operation of QR decomposition (in file mkl_qr.c all the additional headers are included) is compiled as follows: gcc -c mkl_qr.c gcc -o mkl_qr mkl_qr.o -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_intel_ilp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl What would be the proper Makefile to edit and extend with additional libraries if I want to use MKL in batcalc.c, for example? Thank you and best regards, Oksana

1 0

How can i change plan choice according new constraint
by DEMBELE Simon Pierre 10 Feb '20

10 Feb '20

hello, I want to know if it is possible to change how monetdb choose a plan by adding new contraints. -- Simon

2 2

A bug occurs when query logging is enabled
by Hanfeng Chen 09 Feb '20

09 Feb '20

Hello, The bug can be reproduced on a standard TPC-H database with scale factor 1. A query returns NULL after query logging is enabled. The computer we use has Intel i7-8700K (12 cores) and 32GB RAM. Please see the details below. >> mclient -d tpch1 -t performance Welcome to mclient, the MonetDB/SQL interactive terminal (Nov2019-SP1) Database: MonetDB v11.35.9 (Nov2019-SP1), 'mapi:monetdb://tigger:50000/tpch1' FOLLOW US on https://twitter.com/MonetDB or https://github.com/MonetDB/MonetDB Type \q to quit, \? for a list of available commands auto commit mode: on sql>select sum(l_extendedprice) from lineitem; +------------------------------------------+ | L2 | +==========================================+ | 229577310901.20 | +------------------------------------------+ 1 tuple sql:0.083 opt:1.056 run:5.898 clk:7.763 ms sql>call sys.querylog_enable(); sql>select sum(l_extendedprice) from lineitem; +------------------------------------------+ | L2 | +==========================================+ | null | +------------------------------------------+ 1 tuple sql:0.095 opt:1.312 run:15.920 clk:25.033 ms Hanfeng Chen

2 1

Re: pymonetdb issue
by alexo15＠comcast.net 04 Feb '20

04 Feb '20

That would be great Thank you Pedro Sent from Xfinity Connect App ------ Original Message ------ From: ferreira To: ALEX OSSIPOV, Communication channel for MonetDB users Sent: February 4, 2020 at 1:42 AM Subject: Re: pymonetdb issue Hello Alex, The pymonetdb developer merged a patch with IPv6 support, which will be available at pymonetdb's next release. As for the R client, it is maintained by a volunteer that has not been active in the MonetDB community lately. I can ask him for it. Beset regards, Pedro On 2/1/20 1:28 AM, ALEX OSSIPOV wrote: Hi Pedro I also found R code has the same problem. I hope it will be addressed in upcoming release as well. Thanks Alex On January 7, 2020 at 9:22 AM ferreira <pedro.ferreira(a)monetdbsolutions.com> wrote: Hello Alex, I just looked at pymonetdb source code and indeed it does not support IPv6 yet. I'm going to create an issue to request for it. The reason is because in the past, the MAPI protocol (which MonetDB uses for socket connections) didn't support IPv6, so the clients (Python, Ruby, JDBC, etc.) never had to implemented them. However they can now add this option. Best regards, Pedro On 12/22/19 5:30 AM, ALEX OSSIPOV wrote: Hi All I installed pymonetdb v 1.2.1 and trying to connect to remote server MonetDB v11.35.3 (Nov2019) listening on IPV6 only. I have got next error in python client: >>> mapi_connection.connect(hostname="<server_name>", port=xxxxx, username="monetdb", password="monetdb", database="demo", language="sql", unix_socket=None, connect_timeout=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/site-packages/pymonetdb/mapi.py", line 143, in connect self.socket.connect((hostname, port)) socket.gaierror: [Errno -2] Name or service not known >>> My pc client is also using ipv6 to connect to MonetDB server. I also have jdbc connection that works fine. Question: Does it mean pymonetdb does not support ipv6 only environment? What would be the reason if it has nothing to do with protocol? Thanks Alex _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

pymonetdb issue
by ALEX OSSIPOV 04 Feb '20

04 Feb '20

Hi All I installed pymonetdb v 1.2.1 and trying to connect to remote server MonetDB v11.35.3 (Nov2019) listening on IPV6 only. I have got next error in python client: >>> mapi_connection.connect(hostname="<server_name>", port=xxxxx, username="monetdb", password="monetdb", database="demo", language="sql", unix_socket=None, connect_timeout=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/site-packages/pymonetdb/mapi.py", line 143, in connect self.socket.connect((hostname, port)) socket.gaierror: [Errno -2] Name or service not known >>> My pc client is also using ipv6 to connect to MonetDB server. I also have jdbc connection that works fine. Question: Does it mean pymonetdb does not support ipv6 only environment? What would be the reason if it has nothing to do with protocol? Thanks Alex

2 3

Python3 packages for epel
by ALEX OSSIPOV 31 Jan '20

31 Jan '20

I cannot seem find MonetDB-python packages in MonetDB EPEL 7/8 directories. Does it mean it is still work in progress ? Thanks Alex

2 2

Logs on Windows
by Илья Хайбуллин 27 Jan '20

27 Jan '20

Sometimes the MonetDB server just shuts down. I want to find at least some information about what happens before the shutdown. Please tell me how you can turn on logs on Windows or in some other way see what happens before shutting down. MonetDB 5 server 11.35.9 (Nov2019-SP1) (64-bit) Copyright (c) 1993 - July 2008 CWI Copyright (c) August 2008 - 2019 MonetDB B.V., all rights reserved Visit https://www.monetdb.org/ for further information Found 32.0GiB available memory, 8 available cpu cores Libraries: libpcre: 8.41 2017-07-05 openssl: OpenSSL 1.1.1d 10 Sep 2019 libxml2: 2.9.9 Compiled by: monet@WIN2K12R2 (x86_64-pc-winnt) Compilation: cl -GF -W3 -WX -MD -nologo -Ox -Zi -Oi Linking : cl -GF -W3 -WX -MD -nologo -Ox -Zi -Oi

1 0

MonetDB crashing due to out of memory - monetdbd[2247] 1.7 (Aug2018-SP2)
by Suresh Devarakonda 22 Jan '20

22 Jan '20

Thank you Ying and Jenin. I am going to upgrade the MonetDB as you suggested and then go from there. Can you please let me know where can I find the upgrade procedure. Thank you for your help in advance. Suresh Devarakonda 84.51° TDO - Database Support Team D: 513.632.3692 M: 859.512.1839 A: 100 W. 5th Street, Cincinnati, Ohio 45202 Cincinnati | Chicago | New York | Portland Facebook LinkedIn Instagram Twitter Please consider the environment before printing this email. -----Original Message----- From: users-list <users-list-bounces+suresh.devarakonda=8451.com(a)monetdb.org> On Behalf Of users-list-request(a)monetdb.org Sent: Friday, January 17, 2020 6:00 AM To: users-list(a)monetdb.org Subject: users-list Digest, Vol 89, Issue 6 Send users-list mailing list submissions to users-list(a)monetdb.org To subscribe or unsubscribe via the World Wide Web, visit https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.monet… or, via email, send a message with subject or body 'help' to users-list-request(a)monetdb.org You can reach the person managing the list at users-list-owner(a)monetdb.org When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..." Today's Topics: 1. Re: MonetDB crashing due to out of memory - monetdbd[2247] 1.7 (Aug2018-SP2) (Ying Zhang) ---------------------------------------------------------------------- Message: 1 Date: Thu, 16 Jan 2020 12:01:27 +0100 From: Ying Zhang <Y.Zhang(a)cwi.nl> To: Communication channel for MonetDB users <users-list(a)monetdb.org> Subject: Re: MonetDB crashing due to out of memory - monetdbd[2247] 1.7 (Aug2018-SP2) Message-ID: <E6A3FCCA-8023-4EBD-AF7A-CF89F615B2D8(a)cwi.nl> Content-Type: text/plain; charset=utf-8 Hai Suresh, First of all, I?d recommend you to upgrade your MonetDB installation to the most recent one, i.e. Nov2019-SP1. There are many improvements related to your problem since Aug2018-SP2. Also, the version you?re using is no longer supported by the open-source team. Please let us know if upgrade helps. But below, a bit more info. inlined. > On 15 Jan 2020, at 23:44, Suresh Devarakonda <Suresh.Devarakonda(a)8451.com> wrote: > > Hi, > > The Moentdb is crashing every couple of weeks due to out of memory in our environment. Did it really crash, or did MonetDB give error messages and stop executing the query? Crash always indicates a problem in MonetDB, but if it?s the second case, it?s expected. > What I have noticed that MonetDB is eating memory and not releasing it at all once we brought up the MonetDB server. I am not sure if this is how MonetDB works or is it a bug. MonetDB is supposed to free up the memory when it no longer use it, which often happens towards the end of a query execution. For further investigation, we need more information after you have upgraded to Nov2019-SP1. In general, we?ll need information with which we can reproduce the problem, e.g. data + queries. > This is running on monetdbd[2247] 1.7 (Aug2018-SP2) version. Also wondering is there way to limit the memory usage of MonetDB. Please beware that depending on your data sizes and (concurrent) queries, MonetDB might run faster into out-of-memory problem, if you limit the amount of memory it may use, How about putting MonetDB in a VM? There are also some undocumented kernel configurations for this, which we don?t recommend to our users lightly? > > Can you guys help me to resolve this issue. If you need professional support, please contact the supporting company MonetDB Solutions. Regards, Jennie > > [root@xxx ~]# monetdb status > name state health remarks > INVPRDB R 6h 67% 3d mapi:monetdb://xxxxx:50000/INVPRDB > FSPPRDB R 6h 50% 6w mapi:monetdb://xxxxx:50000/FSPPRDB > [root@xxxxx ~]# free -g --- Current utilization > total used free shared buff/cache available > Mem: 31 0 22 0 7 30 > Swap: 3 0 3 > > > <image003.png> > > > > > > <image002.jpg> > Suresh Devarakonda > 84.51? > TDO - Database Support Team > D: 513.632.3692 > M: 859.512.1839 > A: 100 W. 5th Street, Cincinnati, Ohio 45202 > > Cincinnati | Chicago | New York | Portland > Facebook LinkedIn Instagram Twitter > Please consider the environment before printing this email. > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww. > monetdb.org%2Fmailman%2Flistinfo%2Fusers-list&data=02%7C01%7Csures > h.devarakonda%408451.com%7Cae946815e409491cb83d08d79b3c9c2c%7C5f9dc6bd > f38a454a864cc803691193c5%7C0%7C0%7C637148556952386018&sdata=3JmWyH > MNaeMNrBp8vX%2FAYES%2By%2FDqNHqGmE%2FKX%2F%2F4m%2Bs%3D&reserved=0 ------------------------------ Subject: Digest Footer _______________________________________________ users-list mailing list users-list(a)monetdb.org https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.monet… ------------------------------ End of users-list Digest, Vol 89, Issue 6 *****************************************

2 2

Re: embedded python in latest release
by alexo15＠comcast.net 21 Jan '20

21 Jan '20

That was it. I was missing python packages Thank you Panos ------ Original Message ------ From: Panagiotis Koutsourakis To: users-list(a)monetdb.org Sent: January 21, 2020 at 7:54 AM Subject: Re: embedded python in latest release Hello Alex, make sure that you have the MonetDB-python or MonetDB-python3 packages installed: dnf install MonetDB-python or dnf install MonetDB-python3 Also you will need to notify the MonetDB server that you want to start with embedded python support. If you are starting the server using the MonetDB daemon, you need to set the relevant property for your database: monetdb set embedpy=true <database> or monetdb set embedpy3=true <database> If you are starting the server by hand, then mserver5 --dbpath=/path/to/database --set embedded_py=2 or mserver5 --dbpath=/path/to/database --set embedded_py=3 Best regards, Panos. On 1/21/20 5:11 AM, ALEX OSSIPOV wrote: Roberto I have all mentioned packages installed. I do not have any issues with R though. Centos based instance also works fine. Thanks Alex On January 20, 2020 at 3:49 AM Roberto Cornacchia <roberto.cornacchia(a)gmail.com> wrote: Alex, Do you have packages python3-devel python3-numpy (python2-devel python2-numpy for python 2) installed in your Fedora system? Yes, you can import what you like in the body of your function, as long as the package is installed in your system. Roberto On Sun, 19 Jan 2020 at 00:12, ALEX OSSIPOV < alexo15(a)comcast.net mailto:alexo15@comcast.net > wrote: I am having problem enabling embedded python in latest realease, MonetDB 5 server v11.35.9 (Nov2019-SP1) My 64bit Fedora environment has multiple versions of both python 2 and 3 wih 32 and 64 bit libs. MonetDB was installed following https://www.monetdb.org/downloads/Fedora/ with additional SELinux. I tried random_integers function from documentation with both embedpy=true and embedpy3=true separately but getting the same type of error: SQL Error [22000]: TypeException:user.s10_0[9]:'pyapi3.eval' undefined in: X_10:bat[:int] := pyapi3.eval(0x559cb0440270:ptr, "{ \n return numpy.random.randint(low, high, size=(amount,))\n}\n;":str, X_7:int, X_8:int, X_9:int); Both py2 and py3 versions work fine on my computer. How do I properly point to a specific python version to be used by MonetDB? Are there any specific env variables that should be set prior to installation/db creation? I tried to define PYTHONPATH but it did not help. Aside question: Can additional modules be imported in function body for PYTHON type functions or does it only work with numpy? Thanks Alex _______________________________________________ users-list mailing list users-list(a)monetdb.org mailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

embedded python in latest release
by ALEX OSSIPOV 21 Jan '20

21 Jan '20

I am having problem enabling embedded python in latest realease, MonetDB 5 server v11.35.9 (Nov2019-SP1) My 64bit Fedora environment has multiple versions of both python 2 and 3 wih 32 and 64 bit libs. MonetDB was installed following https://www.monetdb.org/downloads/Fedora/ with additional SELinux. I tried random_integers function from documentation with both embedpy=true and embedpy3=true separately but getting the same type of error: SQL Error [22000]: TypeException:user.s10_0[9]:'pyapi3.eval' undefined in: X_10:bat[:int] := pyapi3.eval(0x559cb0440270:ptr, "{ \n return numpy.random.randint(low, high, size=(amount,))\n}\n;":str, X_7:int, X_8:int, X_9:int); Both py2 and py3 versions work fine on my computer. How do I properly point to a specific python version to be used by MonetDB? Are there any specific env variables that should be set prior to installation/db creation? I tried to define PYTHONPATH but it did not help. Aside question: Can additional modules be imported in function body for PYTHON type functions or does it only work with numpy? Thanks Alex

3 3

MonetDB crashing due to out of memory - monetdbd[2247] 1.7 (Aug2018-SP2)
by Suresh Devarakonda 16 Jan '20

16 Jan '20

Hi, The Moentdb is crashing every couple of weeks due to out of memory in our environment. What I have noticed that MonetDB is eating memory and not releasing it at all once we brought up the MonetDB server. I am not sure if this is how MonetDB works or is it a bug. This is running on monetdbd[2247] 1.7 (Aug2018-SP2) version. Also wondering is there way to limit the memory usage of MonetDB. Can you guys help me to resolve this issue. [root@xxx ~]# monetdb status name state health remarks INVPRDB R 6h 67% 3d mapi:monetdb://xxxxx:50000/INVPRDB FSPPRDB R 6h 50% 6w mapi:monetdb://xxxxx:50000/FSPPRDB [root@xxxxx ~]# free -g --- Current utilization total used free shared buff/cache available Mem: 31 0 22 0 7 30 Swap: 3 0 3 [cid:image003.png@01D5CBCA.6268DF10] [cid:image002.jpg@01D3E7B3.57185110] Suresh Devarakonda 84.51<http://www.8451.com/>° TDO - Database Support Team D: 513.632.3692 M: 859.512.1839 A: 100 W. 5th Street, Cincinnati, Ohio 45202 Cincinnati | Chicago | New York | Portland Facebook<https://www.facebook.com/8451group/?ref=aymt_homepage_panel> LinkedIn<https://www.linkedin.com/company/84-51/?trk=biz-companies-cym> Instagram<https://www.instagram.com/84.51/> Twitter<https://twitter.com/8451group> Please consider the environment before printing this email.

2 1

Monetdb crashed issue
by Kalaiarasi Gowtham 14 Jan '20

14 Jan '20

Hi, Currently, we are trying to start the database created in monetdb using the following command monetdb start <dbname> starting database 'dbname'... FAILED start: starting '<dbname>' failed: database '<dbname>' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Getting logs as follows [root@localhost mydbfarm]# tail -f merovingian.log 2020-01-14 14:56:20 MSG merovingian[19527]: database '<dbname>' (19757) has exited with exit status 1 2020-01-14 14:56:20 ERR control[19527]: !monetdbd: an internal error has occurred 'database '<dbname>' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details' 2020-01-14 14:56:25 ERR merovingian[19527]: client error: database '<dbname>' has crashed after starting, manual intervention needed, check monetdbd's logfile (merovingian.log) for details Please,help us to solve this issue. Thanks Kalai

2 1