- users-list - monetdb.org

Questions
by Bouslah Ahmed 10 Mar '13

10 Mar '13

Hi, I am a university student and I use MonetDB. I have two questions to ask you regarding loading data using mclient: 1 - Is it possible to load multiple csv files into multiple tables in one query? For exemple i have five csv files and five tables and I want to use a single command line instead of five mclient -u us1 -d demo -s " COPY INTO DWDATE FROM STDIN USING DELIMITERS '|','\n'" - <F:\2Go\DATE.tbl mclient -u us1 -d demo -s " COPY INTO CUSTOMER FROM STDIN USING DELIMITERS '|','\n'" - <F:\2Go\CUSTOMER.tbl mclient -u us1 -d demo -s " COPY INTO SUPPLIER FROM STDIN USING DELIMITERS '|','\n'" - <F:\2Go\SUPPLIER.tbl mclient -u us1 -d demo -s " COPY INTO PART FROM STDIN USING DELIMITERS '|','\n'" - <F:\2Go\PART.tbl mclient -u us1 -d demo -s " COPY INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" -<F:\2Go\LINEORDER.tbl 2 - Is it possible to enter the password in the request as well as user and database name? This will save me introduce each time the password For exemple mclient -u us1 -d demo -s " COPY INTO DWDATE FROM STDIN USING DELIMITERS '|','\n'" - <F:\2Go\DATE.tbl <PassWord> thank you.

1 0

Parse dates in CSV file, CSV vault
by Aubrey Barnard 10 Mar '13

10 Mar '13

MonetDB Users, Is YYYY-MM-DD the only date format that can be parsed and read from a CSV with the COPY command? Or can the format be specified? On a related note, is the CSV vault feature ready for use (Feb2013 version)? The mailing list implies it is not ready and there is nothing in the release notes. If it is ready, how do I set it up? Thanks much for helping me get my data loaded. Aubrey Barnard

2 1

RE: Installation on Ubuntu 12.10 Quantal
by Duncan Garmonsway 10 Mar '13

10 Mar '13

Thanks, that worked. Didn't realise I was still such a noob... Cheers, Duncan On 10 March 2013 00:00, <users-list-request(a)monetdb.org> wrote: > Send users-list mailing list submissions to > users-list(a)monetdb.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.monetdb.org/mailman/listinfo/users-list > or, via email, send a message with subject or body 'help' to > users-list-request(a)monetdb.org > > You can reach the person managing the list at > users-list-owner(a)monetdb.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users-list digest..." > > > Today's Topics: > > 1. INSERT INTO ... SELECT * FROM ... OFFSET X seems to leave > destination table in an inconsistent state (Percy Wegmann) > 2. Re: INSERT INTO ... SELECT * FROM ... OFFSET X seems to leave > destination table in an inconsistent state (Martin Kersten) > 3. Installation on Ubuntu 12.10 Quantal (Duncan Garmonsway) > 4. Re: Installation on Ubuntu 12.10 Quantal (Brandon Jackson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Mar 2013 10:22:53 -0600 > From: Percy Wegmann <percy(a)clicksecurity.com> > To: Communication channel for MonetDB users <users-list(a)monetdb.org> > Subject: INSERT INTO ... SELECT * FROM ... OFFSET X seems to leave > destination table in an inconsistent state > Message-ID: > < > CADpZG+Scab3V8u_isKziJobUgCYO4BEmkCWMB8iOD-pjggTWQQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Dear Monet Team, > > Today I ran into an issue with copying data between tables. I've tried it > on 11.13.7 and 11.13.9 with the same results. What I find is that when > including an OFFSET clause in my source query, the destination table ends > up in a weird state. > > I've logged a bug for it - http://bugs.monetdb.org/show_bug.cgi?id=3249 > > I'm using the workaround of ordering my data and using a LIMIT instead of > an OFFSET, which seems to work. Just wanted to let you know. > > Thanks, > Percy >

1 0

Installation on Ubuntu 12.10 Quantal
by Duncan Garmonsway 09 Mar '13

09 Mar '13

Hello, After installation I am getting the error: $ monetdbd create ~/monetdb/ monetdbd: error while loading shared libraries: libstream.so.3: cannot open shared object file: No such file or directory But the library (and the symlink to it) are right here: $ ls -l /usr/local/lib | grep libstream -rwxr-xr-x 1 root root 943 Mar 9 14:23 libstream.la lrwxrwxrwx 1 root root 18 Mar 9 14:23 libstream.so -> libstream.so.3.1.5 lrwxrwxrwx 1 root root 18 Mar 9 14:23 libstream.so.3 -> libstream.so.3.1.5 -rwxr-xr-x 1 root root 107879 Mar 9 14:23 libstream.so.3.1.5 And there were no errors reported during any of ./configure sudo make sudo make install My installation script was: sudo apt-get install libxml2-dev bison ant flex unixODBC-dev # download and unpack source wget http://dev.monetdb.org/downloads/sources/Latest/MonetDB-11.15.3.tar.xz tar -xvf MonetDB-11.15.3.tar.xz # install cd MonetDB-11.15.3 ./configure sudo make sudo make install Help much appreciated, Duncan

2 1

INSERT INTO ... SELECT * FROM ... OFFSET X seems to leave destination table in an inconsistent state
by Percy Wegmann 08 Mar '13

08 Mar '13

Dear Monet Team, Today I ran into an issue with copying data between tables. I've tried it on 11.13.7 and 11.13.9 with the same results. What I find is that when including an OFFSET clause in my source query, the destination table ends up in a weird state. I've logged a bug for it - http://bugs.monetdb.org/show_bug.cgi?id=3249 I'm using the workaround of ordering my data and using a LIMIT instead of an OFFSET, which seems to work. Just wanted to let you know. Thanks, Percy

2 1

Announcement: New Feb2013-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 07 Mar '13

07 Mar '13

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2013-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Feb 2013-SP1 bugfix release Bug Fixes * 3228: Python client - mapi.Connection.__getbyes should throw exception when 0 bytes received from socket * 3229: GRANT does not work for UPDATE * 3230: GRANT SELECT priveleges not set in SQL FUNCTIONs * 3233: getting a crash of server with segmentation fault on SLES 11 64 SP2 * 3235: MapiPP.pm unquote '0' as 'undef' * 3236: mclient \d produces no output * 3237: Assertion on BATgroup_internal * 3241: SEGFAULT on string processing when using optimizer.reduce()

1 0

good new monetdb question on stackoverflow
by Anthony Damico 28 Feb '13

28 Feb '13

http://stackoverflow.com/questions/15136868/why-is-group-by-giving-only-one… I couldn't figure out the answer..

2 1

Investigating Segfault
by Percy Wegmann 25 Feb '13

25 Feb '13

Dear Monet Team, We're having an issue in which certain data is causing mserver5 to crash. After this condition is hit, mserver5 crashes at every startup and always dumps an identical core. We're running Monet 11.13.7 on Ubuntu Linux 64 bit. The error is happening on line 1142 of gdk_atoms.c: if (GDK_STRCMP(v, (str) (next + 1) + extralen) == 0) { Examining the core dump revealed that (next + 1) + extralen is referring to an out of bounds address. Here's the backtrace: #0 0x00007faf58414829 in strPut (h=0x1e2d180, dst=0x7fff592cf8f8, v=0x314dac0 "SAD014H1") at gdk_atoms.c:1142 #1 0x00007faf582dc935 in BATappend (b=0x1e2cf90, n=0x32dfdb0, force=1 '\001') at gdk_batop.c:578 #2 0x00007faf584c301e in la_bat_updates (lg=0x2d9b030, la=0x2c3ef48) at gdk_logger.c:429 #3 0x00007faf584c3cf9 in la_apply (lg=0x2d9b030, c=0x2c3ef48) at gdk_logger.c:645 #4 0x00007faf584c3f26 in tr_commit (lg=0x2d9b030, tr=0x2e247d0) at gdk_logger.c:705 #5 0x00007faf584c4533 in logger_readlog (lg=0x2d9b030, filename=0x7fff592d1e80 "/opt/clicksecurity/data/_monetdb/click/sql_logs/sql/log.56") at gdk_logger.c:823 #6 0x00007faf584c482a in logger_readlogs (lg=0x2d9b030, fp=0x2d9b160, filename=0x7fff592d3f90 "/opt/clicksecurity/data/_monetdb/click/sql_logs/sql/log") at gdk_logger.c:896 #7 0x00007faf584c6f3e in logger_new (debug=0, fn=0x7faf500adfa8 "sql", logdir=0x7faf50090a08 "sql_logs", dbname=0x1fa3da0 "click", version=52001, prefuncp=0x7faf500746a1 <bl_preversion>, postfuncp=0x7faf500747ed <bl_postversion>) at gdk_logger.c:1420 #8 0x00007faf584c704e in logger_create (debug=0, fn=0x7faf500adfa8 "sql", logdir=0x7faf50090a08 "sql_logs", dbname=0x1fa3da0 "click", version=52001, prefuncp=0x7faf500746a1 <bl_preversion>, postfuncp=0x7faf500747ed <bl_postversion>) at gdk_logger.c:1446 #9 0x00007faf50075b19 in bl_create (logdir=0x7faf50090a08 "sql_logs", dbname=0x1fa3da0 "click", cat_version=52001) at bat_logger.c:249 #10 0x00007faf50060ce4 in store_init (debug=0, store=store_bat, logdir=0x7faf50090a08 "sql_logs", dbname=0x1fa3da0 "click", stk=0) at store.c:1287 #11 0x00007faf4ffe3d3c in mvc_init (dbname=0x1fa3da0 "click", debug=0, store=store_bat, stk=0) at sql_mvc.c:51 #12 0x00007faf4ff66874 in SQLinit () at sql_scenario.c:230 #13 0x00007faf4ff6651f in SQLprelude () at sql_scenario.c:159 #14 0x00007faf58b3085d in malCommandCall (stk=0x2d36e80, pci=0x2ea5520) at mal_interpreter.c:137 #15 0x00007faf58b331b5 in runMALsequence (cntxt=0x7faf5988c020, mb=0x1e04310, startpc=1, stoppc=0, stk=0x2d36e80, env=0x0, pcicaller=0x0) at mal_interpreter.c:710 #16 0x00007faf58b323c1 in runMAL (cntxt=0x7faf5988c020, mb=0x1e04310, startpc=1, mbcaller=0x0, env=0x0, pcicaller=0x0) at mal_interpreter.c:454 #17 0x00007faf58b60a08 in MALengine (c=0x7faf5988c020) at mal_session.c:619 #18 0x00007faf58b5f21f in malBootstrap () at mal_session.c:64 #19 0x00007faf58b1313b in mal_init () at mal.c:244 #20 0x000000000040340e in main (argc=22, av=0x7fff592db568) at mserver5.c:582 I'm digging into it now, but I was hoping that it might ring some bells. Thanks, -- Percy Wegmann +1 512 637 8500 ext 148

2 3

MonetDB Feb2013 Release --dbpath option
by Gennadiy 22 Feb '13

22 Feb '13

Hello, Before MonetDB Feb2013 Release, I used to start MonetDB using the following Windows command: MSQLserver.bat --dbname=MyNewDatabase MSQLserver.bat called M5server.bat, M5server.bat set --dbfarm option. The result was that database with name specified with --dbname option was created in the directory specified with --dbfarm option. MonetDB Feb2013 Release replaced --dbname and --dbfarm options with single --dbpath option. This --dbpath option is set to some_path\dbfarm\demo in M5server.bat. So, now (MonetDB Feb2013 Release) when I start MonetDB with the following Windows command: MSQLserver.bat --dbpath=some_path\MyNewDatabase MonetDB ignores my --dbpath option, because it uses --dbpath option from M5server.bat! So, how can I start MonetDB and create my own database? Thank you very much, Gennadiy

1 1

Windows Concurrency Error when accessing two separate tables in two separate mclient instances connected to the same mserver?
by Anthony Damico 22 Feb '13

22 Feb '13

Hi, I am using a brand new mserver.exe instance of MonetDB5 on Windows. I am getting the error: "COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead" that I'm not sure I should? Shouldn't I be able to use the COPY command on two separate CSV files into two separate data tables at once? I don't think this behavior is correct, but maybe I'm wrong? :/ If this is incorrect behavior, should I file a bug report for this? The files you can use to re-create this error are: http://downloads.cms.gov/BSAPUF/2008_BSA_PartD_Events_PUF_1.zip http://downloads.cms.gov/BSAPUF/2008_BSA_PartD_Events_PUF_2.zip I create an mserver.exe instance with: # MonetDB 5 server v11.13.9 "Oct2012-SP3" # Serving database 'bsapuf', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 7.860 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50003/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded Then I ran this command in the first mclient: CREATE TABLE bsa1 (PDE_EVENT_ID VARCHAR(255), BENE_SEX_IDENT_CD DOUBLE PRECISION, BENE_AGE_CAT_CD DOUBLE PRECISION, PDE_DRUG_CD DOUBLE PRECISION, PDE_DRUG_STR_CD DOUBLE PRECISION, PDE_DRUG_STR_UNITS_CD DOUBLE PRECISION, PDE_DRUG_DOSE_CD DOUBLE PRECISION, PDE_DRUG_CLASS_CD DOUBLE PRECISION, PDE_DRUG_QTY_DIS DOUBLE PRECISION, PDE_DRUG_DAY_SPLY_CD DOUBLE PRECISION, PDE_DRUG_COST DOUBLE PRECISION, PDE_DRUG_PAT_PAY_CD DOUBLE PRECISION, PDE_DRUG_TYPE_CD DOUBLE PRECISION) ; copy 1000000 offset 2 records into bsa1 from 'c:\\temp\\2008_BSA_PartD_Events_PUF_1.csv' using delimiters ',' NULL AS '' ; And (at the same time) ran this command in the second mclient: CREATE TABLE bsa2 (PDE_EVENT_ID VARCHAR(255), BENE_SEX_IDENT_CD DOUBLE PRECISION, BENE_AGE_CAT_CD DOUBLE PRECISION, PDE_DRUG_CD DOUBLE PRECISION, PDE_DRUG_STR_CD DOUBLE PRECISION, PDE_DRUG_STR_UNITS_CD DOUBLE PRECISION, PDE_DRUG_DOSE_CD DOUBLE PRECISION, PDE_DRUG_CLASS_CD DOUBLE PRECISION, PDE_DRUG_QTY_DIS DOUBLE PRECISION, PDE_DRUG_DAY_SPLY_CD DOUBLE PRECISION, PDE_DRUG_COST DOUBLE PRECISION, PDE_DRUG_PAT_PAY_CD DOUBLE PRECISION, PDE_DRUG_TYPE_CD DOUBLE PRECISION) ; copy 1000000 offset 2 records into bsa2 from 'c:\\temp\\2008_BSA_PartD_Events_PUF_2.csv' using delimiters ',' NULL AS '' ; Here were the results. One worked, one didn't: Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\anthonyd.KFF>cd .. C:\Users>cd .. C:\>cd "Program Files\MonetDB\MonetDB5" C:\Program Files\MonetDB\MonetDB5>mclient -p 50003 "bsapuf" user(win32):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2012-SP3) Database: MonetDB v11.13.9 (Oct2012-SP3), 'bsapuf' Type \q to quit, \? for a list of available commands auto commit mode: on sql> sql> sql>CREATE TABLE bsa2 (PDE_EVENT_ID VARCHAR(255), BENE_SEX_IDENT_CD DOUBLE PRECI SION, BENE_AGE_CAT_CD DOUBLE PRECISION, PDE_DRUG_CD DOUBLE PRECISION, PDE_DRUG_S TR_CD DOUBLE PRECISION, PDE_DRUG_STR_UNITS_CD DOUBLE PRECISION, PDE_DRUG_DOSE_CD DOUBLE PRECISION, PDE_DRUG_CLASS_CD DOUBLE PRECISION, PDE_DRUG_QTY_DIS DOUBLE P RECISION, PDE_DRUG_DAY_SPLY_CD DOUBLE PRECISION, PDE_DRUG_COST DOUBLE PRECISION, PDE_DRUG_PAT_PAY_CD DOUBLE PRECISION, PDE_DRUG_TYPE_CD DOUBLE PRECISION) ; operation successful (9.511ms) sql> sql>copy 1000000 offset 2 records into bsa2 from 'c:\\temp\\2008_BSA_PartD_Event s_PUF_2.csv' using delimiters ',' NULL AS '' ; 1000000 affected rows (42.4s) sql> XXXXXXXXXXXXXXXXXXXXXXXXXXXXX Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\anthonyd.KFF>cd .. C:\Users>cd .. C:\>cd "Program Files\MonetDB\MonetDB5" C:\Program Files\MonetDB\MonetDB5>mclient -p 50003 "bsapuf" user(win32):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2012-SP3) Database: MonetDB v11.13.9 (Oct2012-SP3), 'bsapuf' Type \q to quit, \? for a list of available commands auto commit mode: on sql>\d sql>CREATE TABLE bsa1 (PDE_EVENT_ID VARCHAR(255), BENE_SEX_IDENT_CD DOUBLE PRECI SION, BENE_AGE_CAT_CD DOUBLE PRECISION, PDE_DRUG_CD DOUBLE PRECISION, PDE_DRUG_S TR_CD DOUBLE PRECISION, PDE_DRUG_STR_UNITS_CD DOUBLE PRECISION, PDE_DRUG_DOSE_CD DOUBLE PRECISION, PDE_DRUG_CLASS_CD DOUBLE PRECISION, PDE_DRUG_QTY_DIS DOUBLE P RECISION, PDE_DRUG_DAY_SPLY_CD DOUBLE PRECISION, PDE_DRUG_COST DOUBLE PRECISION, PDE_DRUG_PAT_PAY_CD DOUBLE PRECISION, PDE_DRUG_TYPE_CD DOUBLE PRECISION) ; operation successful (8.636ms) sql> sql>copy 1000000 offset 2 records into bsa1 from 'c:\\temp\\2008_BSA_PartD_Event s_PUF_1.csv' using delimiters ',' NULL AS '' ; 1000000 affected rows (41.6s) COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK in stead sql> sql> Thanks as always! Anthony

2 2