- developers-list - monetdb.org

New Dec2016 stable branch created
by Sjoerd Mullender 14 Oct '16

14 Oct '16

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Dec2016. So in each of your stable checkout directories you can do hg pull hg update -rDec2016 to update to this new branch. -- Sjoerd Mullender

1 0

Announcement: New Jun2016-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 13 Oct '16

13 Oct '16

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2016-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jun 2016-SP2 bugfix release Build Environment * We now use the CommonCrypto library instead of the OpenSSL library on Darwin. Bug Fixes * 3939: Assert failure on concurrent queries when querying sys.queue * 4019: Casting a timestamp from a string results in NULL * 4025: expressions in the WHERE clause that evaluates incorrectly * 4038: After upgrade from 11.21.19, jdbc couldn't list tables for non sys users * 4044: Server crash when trying to delete a table has been added to a merge table with "cascade" at the end * 4049: Wrong results for queries with "OR" and "LEFT JOIN" * 4052: Infinite loop in rel_select * 4054: copy into file wrongly exports functions * 4058: Server crashes with a particular conditional query * 4059: Geom functions only visible by user monetdb * 4060: BAT leak in some aggregate queries * 4062: Error: SELECT: no such binary operator 1800 790 9186'like(varchar,varchar)' when used in query running in other schema than sys * 4064: Assertion: column not found * 4067: Relevant column name not printed when a CSV parsing error occurs * 4070: Extra condition in join predicate of explicit join produces wrong MAL code * 4074: Cannot use prepared statements when caching disabled * 6065: CTE with row number and union fails within MAL

1 0

Bulk load from memory
by Hermano ... 09 Oct '16

09 Oct '16

Hello I became very interested in MonetDB during my research, and I am considering developing my own MAL compiler for a system based on a data model my research group is working on in the future. For now, I've been trying to understand MAL and how the COPY INTO command works in MonetDB Initially, I wish to do a large bulk load from an array in memory directly into MonetDB. My first "closed box" approach was to do the following: -Create a binary integer file containing milions of integers in /dev/shm/ named teste.bin. It is a memory based device, so the data is expected to be in main memory. -Create a table in MonetDB: CREATE TABLE teste (val INT); -Try to do BINARY COPY FROM: COPY BINARY INTO teste FROM ('/dev/shm/teste.bin'). When I try to this I get the error: Failed to attach file /dev/shm/teste.bin BATattach: cannot rename heapfile !OS: Invalid cross-device link After that I understood that monetdb basically moves the entire file around, which can't be done with data from another device. I wish to know if there is any sort of functionality in MonetDB or even a MAL module that would allow me to read from some sort of shared memory buffer (kind vague I know) directly into a BAT file. The MAL program for the COPY command has an sql.importTable and sql.append calls, so I wonder if there is some variant of those that would allow me to load binary data from a memory buffer? Thanks in Advance

1 0

Fwd: lib_pyapi complains about undefined symbol PyInt_Type
by Dean De Leo 06 Oct '16

06 Oct '16

Hi, on my personal machine, when starting mserver5, monetdb complains with the following message: !SyntaxException:parseError: !LoaderException:loadLibrary:Loading error failed to open library pyapi (from within file '/linux/workspace/monetdb/build/lib/monetdb5/lib_pyapi.so'): /linux/workspace/monetdb/build/lib/monetdb5/lib_pyapi.so: undefined symbol: PyInt_Type !SyntaxException:parseError: !<identifier> expected I guess this is another case of conflict Python 2 vs 3. I am using ArchLinux where the default is v3. In my case a workaround seems to be forcing back python2 (see patch). Dean

2 1

GDK_mmap_minsize again
by Hannes Mühleisen 24 Sep '16

24 Sep '16

Hello list, we were wondering about the purpose of GDK_mmap_minsize when creating transient columns. The attached patch will always *try* to malloc/realloc a transient column but still fall back to memory-mapped files if malloc should fail. This dramatically improves performance. Any good reason why this should not be the default behaviour? Thanks, Mark and Hannes

5 5

eviction in Monetdb
by Narendra Pradeep 15 Sep '16

15 Sep '16

Hello , Need to know about monetdb eviction policy ? My understanding is Monetdb creates a buffer pool {BBP} and inserts all the bats that are used . But not sure when does Monetdb deletes the bat from the buffer pool and what eviction policy does it use (like LRU or based on some threshold) ? Thanks

2 1

Re: MonetDB: graph1 - Add the operator bat.getCount() to retrieve th...
by Stefan Manegold 14 Sep '16

14 Sep '16

Hi Dean, how is bat.getCount() different from aggr.count() ? ;-) Thanks! Stefan ----- On Sep 14, 2016, at 6:30 PM, Dean De Leo commits+dleo=cwi.nl(a)monetdb.org wrote: > Changeset: 9ac831deacb3 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9ac831deacb3 > Modified Files: > monetdb5/modules/kernel/bat5.c > monetdb5/modules/kernel/bat5.h > monetdb5/modules/kernel/bat5.mal > Branch: graph1 > Log Message: > > Add the operator bat.getCount() to retrieve the cardinality of a bat > > > diffs (48 lines): > > diff --git a/monetdb5/modules/kernel/bat5.c b/monetdb5/modules/kernel/bat5.c > --- a/monetdb5/modules/kernel/bat5.c > +++ b/monetdb5/modules/kernel/bat5.c > @@ -496,6 +496,19 @@ BKCgetColumnType(str *res, const bat *bi > } > > str > +BKCgetCount(lng *res, const bat *bid) > +{ > + *res = lng_nil; > + if (BBPcheck(*bid, "bat.getCount")) { > + BAT *b = BBPquickdesc(*bid, 0); > + > + if (b != NULL) > + *res = (lng) BATcount(b); > + } > + return MAL_SUCCEED; > +} > + > +str > BKCgetRole(str *res, const bat *bid) > { > BAT *b; > diff --git a/monetdb5/modules/kernel/bat5.h b/monetdb5/modules/kernel/bat5.h > --- a/monetdb5/modules/kernel/bat5.h > +++ b/monetdb5/modules/kernel/bat5.h > @@ -29,6 +29,7 @@ mal_export str BKCbun_inplace_force(bat > mal_export str BKCbat_inplace_force(bat *r, const bat *bid, const bat *rid, > const bat *uid, const bit *force); > mal_export str BKCgetCapacity(lng *res, const bat *bid); > mal_export str BKCgetColumnType(str *res, const bat *bid); > +mal_export str BKCgetCount(lng *res, const bat *bid); > mal_export str BKCgetRole(str *res, const bat *bid); > mal_export str BKCsetkey(bat *res, const bat *bid, const bit *param); > mal_export str BKCisSorted(bit *res, const bat *bid); > diff --git a/monetdb5/modules/kernel/bat5.mal b/monetdb5/modules/kernel/bat5.mal > --- a/monetdb5/modules/kernel/bat5.mal > +++ b/monetdb5/modules/kernel/bat5.mal > @@ -77,6 +77,10 @@ command getCapacity(b:bat[:any_1]):lng > address BKCgetCapacity > comment "Returns the current allocation size (in max number of elements) of a > BAT."; > > +command getCount(b:bat[:any_1]):lng > +address BKCgetCount > +comment "Returns the current tuple count of a BAT."; > + > command getColumnType( b:bat[:any_1] ) :str > address BKCgetColumnType > comment "Returns the type of the tail column of a BAT, as an integer type > number."; > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

BAT in memory
by Narendra Pradeep 30 Aug '16

30 Aug '16

Hello , I am reading monetdb source code . I need to know which function check whether the bat of the particular column lies in memory or not ? And how does it maps the bat with corresponding column ? Thanks . Regars , Naren

1 0

Monetdb printing values of variables in console or terminal
by Narendra Pradeep 23 Aug '16

23 Aug '16

Hai everyone , How to print messages in console or to write something in log file of the Monetdb . can someone help me with exact function ? I cant even write a word in logfile and console is there something with my build ? Regards , Pradeep

1 0

compression in monetdb
by Narendra Pradeep 11 Aug '16

11 Aug '16

Hai , Read that dictionary encoding occurs in monetdb . so 1) what is dictionary compression technique ? 2) Does it works for strings alone or other data structures too ? 3) I a mailing list it was stated that it works only till the size of the column in 128 MB for older release? Is this the same for current release ? 4) Does Monetdb uses any other compression technique ? 5) If yes where can I find the source of it (Precisely , which file in source ) Thanks in advance regards , Pradeep

1 0