- developers-list - monetdb.org

[Monetdb-developers] Some info
by luca paganotti 22 Nov '11

22 Nov '11

Hi all, want to try monetdb C API. I've read the examples including Mapi.h from mapilib directory. I do not find any mapilib dir. On windows Mapi.h is in the lib dir under monetdb root. On linux Mapi.h is in /usr/include/monetdb. Are the examples updated with the last monetdb distribution? I've read Mapi.h. Mapi type is defined as MapiStruct* but I can't find any definition for MapiStruct in Mapi.h. The same holds for MapiHdl vs MapiStatement*. Am I missing something? Thank you for any answer.

2 2

[Monetdb-developers] Announcement: New Aug2011-SP3 Bugfix release of MonetDB suite
by Fabian Groffen 22 Nov '11

22 Nov '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP3 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Note that this is the last release of the Aug2011 branch. From now on, no further development will take place on this branch. Aug 2011-SP3 bugfix release SQL * Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements. Merovingian * Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port. Bug Fixes * 2875: incomplete database initialization * 2912: Function body disappears after loading data - error message: SQLException:sql.resultSet:Cannot access descriptor * 2919: Add column and select in same transaction * 2920: Updated database from Aug2011 to Dec2011 make queries fail with "Subquery result missing". * 2921: complex where clause broken * 2923: Multiplication and cast from decimal to integer leads to 0 value * 2924: use LIKE with ESCAPE ignores OR operator * 2928: Cannot update a field with a count query based on an other table

1 0

[Monetdb-developers] Information request
by luca paganotti 21 Nov '11

21 Nov '11

Sorry for insisting, I'm interested in learning more about monetdb, but I've some difficulties in finding the informations I'm searching for. I would like to learn more about the C API, so I'm searching for some coding examples. The only thing I found was presumably an old example (as Stefan told me) about mapi usage that unfortunately doesn't work for me with monetdb current release (tried both linux and windows releases). I wrote two times to the developer list but at the moment no meaningful answer. Am I missing something? Have monetdb a still live developer community.? Should I take a direct contact to info(a)monetdb.org? Should I use bugzilla tracker? Hope not to annoy anyone. Best regards. -------------------------------------------------------------- -- Dott. Ing. Luca Paganotti -- Via dei Giardini 9 -- 21035 Cunardo (VA) -- 393 1346898 -------------------------------------------------------------- -- sourceforge email: -- lucapaganotti(a)users.sourceforge.net -- enml.org email: -- luca.paganotti(a)enml.org -- skype name: luca.paganotti --

4 9

[Monetdb-developers] Fwd: Some info 2
by luca paganotti 17 Nov '11

17 Nov '11

Hi Stefan, I'm viewing the docs on http:// http://www.monetdb.org/Documentation/Manuals/SQLreference/Programming/MAPI, isn't this the right place? Thanks for answering. On Tue, Nov 15, 2011 at 11:40 PM, Stefan Manegold <Stefan.Manegold(a)cwi.nl>wrote: > Hi Luca, > > monetdb-config & monetdb-clients-config are obsolete. > > Which "Mapi example" are you refering to? > > Stefan > > On Tue, Nov 15, 2011 at 11:22:23PM +0100, luca paganotti wrote: > > Hi all, trying to build the Mapi example I can't find monetdb-config > and/or > > monetdb-clients-config. Where do I get them? > > > > Thanks > > > > ------------------------------------------------------------------------------ > > RSA(R) Conference 2012 > > Save $700 by Nov 18 > > Register now > > http://p.sf.net/sfu/rsa-sfdev2dev1 > > > _______________________________________________ > > Monetdb-developers mailing list > > Monetdb-developers(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > -- > | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | > | http://CWI.nl/~manegold/ | Science Park 123 (L321) | > | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >

1 0

[Monetdb-developers] Some info 2
by luca paganotti 15 Nov '11

15 Nov '11

Hi all, trying to build the Mapi example I can't find monetdb-config and/or monetdb-clients-config. Where do I get them? Thanks

2 1

[Monetdb-developers] Announcement: New Aug2011-SP2 Bugfix release of MonetDB suite
by MonetDB Release Engineer 26 Oct '11

26 Oct '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Aug 2011-SP2 bugfix release Client Package * ODBC: Implemented a workaround in SQLTables for bug 2908. * Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability. * Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901 * mclient now checks the result of encoding conversions using the iconv library. * Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv). * Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois. * Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output Merovingian * Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped. * monetdbd get status now also reports the version of the running monetdbd * Improved response time of 'monetdb start' when the database fails to start. * Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database. * To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client. Bug Fixes * 2346: mkey.bulk_rotate_xor_hash MAL exception * 2712: un-rolled-back savepoints cause crash upon scenario exit * 2774: COPY INTO of double (~1.0e-310) cannot be imported. * 2890: Slow execution of COUNT in combination with DISTINCT * 2894: No consistent behavior of implicit cast of integer to double * 2897: Mapi.pm: short reads not handled when reading blocks from server * 2898: order by for multiple (two) columns fails for large tables when second ordering is 'desc': "mat.refine_reverse' undefined" * 2900: rand(int) should not return NULL * 2901: Make keyword arguments in Python DB API connect() function more standard * 2904: TypeException after calling UDF in other than sys schema (when it is initialised) * 2906: "ALTER TABLE mytable SET READ ONLY" results in server segfault * 2908: SQL query produces error with MAL statements * 2909: query result is not displayed when history = true

1 0

[Monetdb-developers] New Dec2011 stable branch created
by Sjoerd Mullender 17 Oct '11

17 Oct '11

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Dec2011. So in each of your stable checkout directories you can do hg pull hg update -rDec2011 to update to this new branch. -- Sjoerd Mullender

1 0

[Monetdb-developers] Date and Time Functions
by Roula Avg 15 Oct '11

15 Oct '11

Hello, i am looking for a manual about all the date and time functions supported in Monetdb. In the official site there is no such manual, can anyone help me please? Thank you very much

1 0

[Monetdb-developers] MonetDB bug?
by Arjen P. de Vries 14 Oct '11

14 Oct '11

Dear all, I took from http://www.monetdb.org/Documentation/Manuals/SQLreference/TableMgmt that alter table MY-TABLE-NAME set read only should work. In my Apr-SP1 version on CentOS this however crashes mclient. Vriendelijke groet, Arjen -- ==================================================================== CWI, room L333 Centre for Mathematics and Computer Science Science Park 123 Email: Arjen.de.Vries(a)cwi.nl 1098 XG Amsterdam tel: +31-(0)20-5924306 The Netherlands fax: +31-(0)20-5924199 ===================== http://www.cwi.nl/~arjen/ ====================

2 1

[Monetdb-developers] Can't start the database after build
by Mark Kerzner 11 Oct '11

11 Oct '11

Hi, I've built the latest code from Mercurial repository, and I cannot get any database to start. I am doing everything locally, so this sequence of commands invariably results in the db crashing: monetdbd start monetdb-farm/ monetdb create demo created database in maintenance mode: demo mark@mark-desktop:~$ monetdb start demo starting database 'demo'... FAILED start: starting 'demo' failed: database 'demo' has crashed after starting, manual intervention needed, check monetdbd's logfile for details and not much more in the log 2011-10-10 19:12:38 MSG control[4544]: (local): created database 'demo' 2011-10-10 19:12:48 MSG control[4544]: (local): served status list 2011-10-10 19:12:48 MSG merovingian[4544]: startup of database under maintenance 'demo' forced 2011-10-10 19:12:48 MSG merovingian[4544]: starting database 'demo', up min/avg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (0-0=0) 2011-10-10 19:12:48 MSG demo[4553]: arguments: /usr/local/bin/mserver5 --set gdk_dbfarm=/home/mark/monetdb-farm --dbname=demo --set merovingian_uri=mapi:monetdb://mark-desktop:50000/demo --set mapi_open=false --set mapi_port=0 --set mapi_usock=/home/mark/monetdb-farm/demo/.mapi.sock --set monet_vault_key=/home/mark/monetdb-farm/demo/.vaultkey --set max_clients=64 --set monet_daemon=yes 2011-10-10 19:12:48 MSG merovingian[4544]: database 'demo' (4553) was killed by signal SIGSEGV 2011-10-10 19:12:48 ERR control[4544]: (local): failed to fork mserver: database 'demo' has crashed after starting, manual intervention needed, check monetdbd's logfile for details Thank you. Sincerely, Mark

5 6

Re: [Monetdb-developers] MonetDB: Aug2011 - Don't append partial queries inside mapi_quer...
by Fabian Groffen 11 Oct '11

11 Oct '11

On 10-10-2011 22:48:49 +0200, Sjoerd Mullender wrote: > Don't append partial queries inside mapi_query_part. > When using the construct > mclient -s 'COPY INTO t FROM STDIN ...' < file.csv > mapi_query is called for the first part of the query (on the command > line) which allocates just enough space to remember that query, and > then mapi_query_part is called for each line of the input file, which > assumed the allocated space was 512 bytes, resulting in out-of-bounds > writes. Wouldn't the fix have been to use (computed) ln instead of sz in the else case of the hdl->query check? I think this is the original problem. With your change now, are back to the situation where only the first line of a query is remembered, which is often as useless as not showing any query at all. Consider SELECT *<enter> FROM table<enter> WHERE blabla<enter> ;<enter> > if (hdl->query == NULL) { > - size_t sz = size; > - > - sz = 512; > - hdl->query = malloc(sz + 1); > - assert(hdl->query); > - hdl->query[0] = '\0'; > - strncpy(hdl->query, query, sz); > - hdl->query[sz] = '\0'; > - } else { > - size_t ln = strlen(hdl->query), sz = 512 - ln; > - if (sz > 0) { > - if (size < sz) > - sz = size; > - assert(hdl->query); > - strncat(hdl->query, query, sz); > - } > + hdl->query = strdup(query); > }

2 1

[Monetdb-developers] How to create a binary load files with string values?
by Mark Kerzner 07 Oct '11

07 Oct '11

Hi, I've tried quite a few things (creating the files/strings with 0 at the end, with 0 and \n at the end, fixed length, etc.) but I still get an error sql>copy into sku from ('/home/mark/MonetDbData/myfile); SQLException:sql:failed to attach file /home/mark/MonetDbData/myfile I know that I can load integer values, because that works, so the problem must be in how I am formatting strings for MonetDB binary load. Are there any examples or any dumps to look at? Thank you, Mark

3 4

[Monetdb-developers] Announcement: New Aug2011-SP1 bugfix release of MonetDB suite
by Fabian Groffen 30 Sep '11

30 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at http://www.monetdb.org/. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Aug 2011-SP1 bugfix release SQL * Fixed a bug when using default values for interval columns. Bug 2877. * A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882. * A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883. Client Package * Fixed Perl DBD rowcount for larger results, bug #2889 * Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema). * Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois. * Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois. * mclient: fix display of varchar columns with only NULL values. * Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables. MonetDB5 Server * Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888. Bug Fixes * 2865: mserver5 --set gdk_nr_threads=2 --forcemito: deadlock during first SQL client connect on virgin (empty) DB * 2877: assertion failed when creating table with default interval * 2879: A SELECT query fails to produce some results * 2882: Delete of some records don't work on Monetdb * 2883: insert into t (select * from t) bogus in multithreaded execution * 2884: Mapi.pm and GetInfo.pm syntax fixes for array usage * 2885: DBD::monetdb table_info error * 2887: Division by zero error, when should not divide at 0 * 2888: test/null/null_in_case.sql hangs server * 2889: Mapi::getBlock is erasing query count on second call

1 0

Re: [Monetdb-developers] MonetDB: default - The sample module for the monetdb5 server.
by Lefteris 27 Sep '11

27 Sep '11

Roberto, we were also puzzled with which syntax to adopt. The first issue is that the syntax is also dictating in a way implementation (although that sounds not correct:)). A syntax that has a order by rand() implies that you are attaching random numbers to the rows and then sort on that. This sorting is very coslty and it is the worst case since you have random order to begin with. Something like that would cost NlogN where N is the size of rows. Moreover, that syntax implies that you are sampling the base table (or some intermediate result anyway) and then "continue" the query evaluation. however sampling is not a very kind operator and cannot by easily pushed down: a sample of a join is not equal with the join of 2 samples. These are in my view the implications (or benefits!) of such a syntax. So MonetDB could as well support the syntax you propose and should! On the other hand, the syntax SAMPLE we just introduce, is applied solely on the final result set of the query, after or query evaluation is done. The algorithm has a complexity of S, where S is the size of the sample (as opposed to NlogN of the previous). Also , the sample operator has the same semantics of limits, it is an operator that shows a limited (random) portion of the entire (usually computed) result. that means that all joins, predicated etc. have been evaluated before a sample is taken. Of course we would love to have that functionality in sub-queries too, hence the workaround. What is really missing now is a function where the table definition is a parameter, indeed! With this 2 views in mind, I would suggest that both syntax are necessary to exist! Each of them serving a different purpose (and different underline implementation eventually). the other solution would be to allow SAMPLE is subqueries and "depart" from the SQL spirit. What do you think? lefteris On Mon, Sep 26, 2011 at 10:41 AM, Roberto Cornacchia < roberto.cornacchia(a)gmail.com> wrote: > Lefteris, I would like to add something about the syntax (and semantics) of > this feature. > See below > > >> +/* >> + * @- Uniform Sampling. >> + * >> + * A new SQL operator has been added to support sampling the result of a >> query. >> + * The syntax for sampling is: >> + * SELECT ... FROM ... WHERE ... SAMPLE s >> + * >> + * where s if is an integer greater than 1, it defines the number of rows >> to be >> + * in the sample. If s is a double between [0.0,1.0] the it refers to the >> + * percentage of the result to be sampled. That is if s=0.3 then the >> sample >> + * will be 30% the size of the query result. >> + * >> > > It has been discussed already whether this choice or an explicit PERCENT is > more intuitive. > Personally, I don't care much about that. > > >> + * SAMPLE is been treated as LIMIT, ORDER BY, etc., that means that it >> can only >> + * be in the outer most SELECT clause, i.e., SAMPLE cannot appear in a >> + * subquery. > > > This is much more interesting to me. > I see that SAMPLE fits in the same semantics as LIMIT. In fact, you can > think of it as a special LIMIT. > What I don't like about it, is that it then shares the same limitations > (which are absurd, as the whole SQL often is, from my point of view), which > is what you mention above about the subquery. > You suggest a workaround here: > > However, if this is needed, then one may define a function, for >> + * example >> + * >> + * CREATE FUNCTION mysample () >> + * RETURNS TABLE(col a,...) >> + * BEGIN >> + * RETURN >> + * SELECT a,... >> + * FROM name_table >> + * SAMPLE 100; >> + * end; >> + * >> + * and then use function mysample() for example to populate a new table >> with >> + * the sample. E.g., >> + * >> + * INSERT INTO sample_table (SELECT * FROM mysample()); >> + * >> + * > > > To me, writing a function with an hard-coded table name in it is... weird! > Like writing a function that can only do 2+2. > (This could be much better if table identifiers were allowed as function > parameters, a feature that I'd love to see implemented!). > > To this, I much prefer a solution based on the window functions RANK() > OVER() and ROW_NUMBER() OVER() which luckily are implemented in MonetDB, > and the function rand(). > > Your example above becomes a regular query that has no nesting-issue: > > SELECT a,b,c > FROM ( > SELECT a,b,c, ROW_NUMBER() OVER(ORDER BY rand()) as my_rank > FROM name_table > ) as x > where x.my_rank <= 100; > > Replace ROW_NUMBER() with RANK() to allow duplicates in the random sample. > > I'm not comparing the efficiency of the two solutions here, your solution > could well be more efficient. > I'm only comparing them from a usability point of view. > > I noticed that my own usage of the LIMIT clause is becoming less and less > frequent (in favour of the syntax I report here) because of the nesting > issue. > What I am wondering is, how much useful is to introduce a new syntax for > the sampling that has this (important, in my view) limitation? How often > would you need a sample in the outermost select clause only? > > Cheers, > Roberto > > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > >

2 1

[Monetdb-developers] MacPorts Portfiles
by Rémy Chibois 20 Sep '11

20 Sep '11

Hi all, You'll find attached an archive containing two portfiles for compiling and installing MonetDB using the MacPorts system. Can you please have a look at it ? In order to use these portfiles, please refer to: http://guide.macports.org/#development.local-repositories If this is of interest for someone, do you want me to submit this port to the MacPorts Project ? Regards, -- Rémy Chibois

2 3

[Monetdb-developers] Announcement: New Apr2011-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 16 Sep '11

16 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011 feature release of the MonetDB suite of programs. The default width for OID columns in the database is now 64 bits for all 64-bit architecture. In releases up to this release, the default width on 64-bit Windows was still 32 bits. This means that with the current release it is possible to have tables with more than about 2 million rows even on Windows (as long as you use a 64-bit build). More information (including release notes) on this release is available at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New Apr2011-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 29 Jul '11

29 Jul '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011-SP2 bug fix release of the MonetDB suite of programs. This is the first release with support for Fedora 15. Fedora 13 is no longer supported. One important bug fix has to do with columns with string (char/varchar) values. There was a bug that could cause exponential growth of the file holding the data for the column. This bug has been fixed. If you have a database that is much larger than you would expect for the amount of data, this might be the cause. The remedy is to dump and restore that database (or just the affected tables). Apart from using up too much space, this bug didn't have any other consequences. More information (including release notes) on this release is available at <http://www.monetdb.org/OldReleaseNotes/Apr2011>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] New Aug2011 stable branch created
by Sjoerd Mullender 11 Jul '11

11 Jul '11

I have created a new stable branch for the next feature release. This branch was created in order to stabilize the code before the feature release. >From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Aug2011. So in each of your stable checkout directories you can do hg pull hg update -rAug2011 to update to this new branch. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] Tuple reconstruction algorithm
by Stefan Manegold 04 Jul '11

04 Jul '11

Hi Tim, [I prefer to keep the mailing list involved to share the discussion; I hope you don't mind ... ;-) ] On Mon, Jul 04, 2011 at 07:30:36PM +0200, Tim Speier wrote: > Hello Stefan, > > thanks for your fast reply. > > I am especially interested in tuple reconstruction mechanisms in > column-oriented databases. So I'm not a user. I see; please also consider reading our papers (cf., http://repository.cwi.nl/search/searchrepository.php?stheme=INS1) in particular http://repository.cwi.nl/search/fullrecord.php?publnr=14290 http://repository.cwi.nl/search/fullrecord.php?publnr=11117 and/or these PhD these: http://repository.cwi.nl/search/fullrecord.php?publnr=14832 http://repository.cwi.nl/search/fullrecord.php?publnr=14301 http://repository.cwi.nl/search/fullrecord.php?publnr=16706 Basically, "even" in main memory, sequential access is fast, while random access is slow. > > > Further, for any set of BATs the represent the individual columns of a > > relational table, the BATs' "heads" (first/left column) are aligned and > > consist of OIDs that form a dense (i.e., increment == 1) ascending sequence > > of integer numbers starting at a given offset (usually 0). Thus, these OIDs > > can indeed be seen as tuple-/record- identifiers. Be aware, though, that > > these OIDs are only used internally and not available as tuple-/record- > > identifiers in SQL. > > Given their denseness properly, these OIDs are usually not materialized > > (read: physically stored) but rather computed on the fly, and they > > effectively indicate (module offset) the physical position of the respective > > record's attribute values in all BATs involved. > > I am sorry, but it is not yet clear to me. Let me create an example: > A relation R with two attributes X and Y. It consists of the following > tuples: > > +---+-----+ > | X | Y | > +---+-----+ > | A | 300 | > +---+-----+ > | B | 100 | > +---+-----+ > | C | 400 | > +---+-----+ > > Now let's create two BATs: each record gets its own OID and, my naive > approach, map each attribute values to a BAT: > > BAT X BAT Y > +---+---+ +---+-----+ > |OID|VAL| |OID|VAL | > +---+---+ +---+-----+ > | 0 | A | | 0 | 300 | > +---+---+ +---+-----+ > | 1 | B | | 1 | 100 | > +---+---+ +---+-----+ > | 2 | C | | 2 | 400 | > +---+---+ +---+-----+ > > Now I could reconstruct each tuple by its OID and the index. But I would > loose some advantages of the column-oriented databases. Like which? I guess you mean having each column sorted individually? > A more sophisticated approach would be to store values in ascending > order like this: > > BAT X BAT Y > +---+---+ +---+-----+ > |OID|VAL| |OID|VAL | > +---+---+ +---+-----+ > | 0 | A | | 1 | 100 | > +---+---+ +---+-----+ > | 1 | B | | 0 | 300 | > +---+---+ +---+-----+ > | 2 | C | | 2 | 400 | > +---+---+ +---+-----+ > > But how can I now reconstruct the tuples efficient now? > Is there an additional index structure (as you said in > your foregoing email). ? In the first case, tuple reconstruction is cheap (almost "for free"), as columns are physically aligned, and hence, highly efficient in-order positional lookup can be exploited for tuple reconstruction. Obviously, with keeping the columns physically aligned does not allow to order them individually according to their values. Thus without further indeces, value-lookups using binary search are only possible for on sorted column; for all other columns, some secondary index structure would be required. MonetDB only used (automatically) hash-indeces for single-value (point-)lookups, but resorts to (highly optimized) sequential scans for any range lookups on non-sorted columns. In the second case, value-lookups can be done (reasonably) efficient using binary search on each column, however tuple-reconstruction (i.e., OID lookups) can no longer be done with simple highly efficient in-order positional lookups. In that case, MonetDB would (automatically on-the-fly) build a hash-index on the non-ordered (non-dense) OID columns. In general, there is no such thing as a "free lunch" ... ;-) Hope this helps you further --- don't hesitate to keep asking in case you have more questions ... Gruß, Stefan > > I'm very sorry for my innocent questions. > Tim -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

3 2

[Monetdb-developers] Tuple reconstruction algorithm
by Tim Speier 04 Jul '11

04 Jul '11

Dear all, I'm trying to understand the tuple reconstruction algorithm in MonetDB. Maybe someone can give me a short overview to understand it or the place in documentation where I can do some research. As far as I understood, the BAT consists of object-ids and values. Is the object-identifier the same as the record- or tuple-identifier? If it so, is there a special mechanism to position a specific oid in a BAT or must it be scanned to find a oid? And if not, how can I determine the corresponding values in different BATs to reconstruct a tuple? Or am I totally wrong and there is an obvious way to reconstruct tuples from different BATs? Thanks in advance, Tim

2 1

[Monetdb-developers] PhD opening at CWI - Adaptive indexing techniques for modern database systems
by Stratos Idreos 23 Jun '11

23 Jun '11

Subject: Adaptive indexing techniques for modern database systems. The Centrum Wiskunde & Informatica (CWI) has a full time vacancy for a talented PhD candidate in the Database Architectures group (http://www.cwi.nl/en/research-groups/Database-Architectures) The position is in the broad area of adaptive indexing techniques for modern database systems. Adaptive indexing is a major challenge in database research; it tries to completely alleviate the need for indexing and tuning steps leading to database systems that require no physical design set-up and can cope immediately with any workload. This leads to database systems that are instantly usable and thus more applicable to numerous new applications that now consider databases too slow, too complex and too old. This position will focus specifically on adaptive indexing for extremely large databases where traditional indexing represents a major bottleneck. The Database Architectures group is one of the world-wide leading groups in system-oriented database research, e.g., it received the 10-year best paper award at VLDB 2009 for their pioneering research on hardware-aware database technology and the column-store model and the 2011 ACM SIGMOD Jim Gray Dissertation award for the database cracking work. The group has a unique methodology that combines fundamental database research with active participation of all group members in database engineering, centred around the open-source analytical database system MonetDB (http://www.monetdb.org/) An internship at one of the partnering database research labs world-wide is considered part of the PhD track. REQUIREMENTS The PhD candidates must have a master degree in computer science (or a 5 year diploma), (specialisation databases, distributed systems, software engineering) mathematics, or related fields. Preferable qualifications for candidates include proven research talent, system programming skills (C/C++), practical experience with using and implementing database systems, and software development in a team. Candidates are expected to have an excellent command of English, and good academic writing and presentation skills. CENTRUM WISKUNDE & INFORMATICA (CWI) Founded in 1946, the Centrum Wiskunde & Informatica (CWI) is the national research institute for mathematics and computer science in the Netherlands. It is located at the Science Park Amsterdam and is part of the “Netherlands Organisation for Scientific Research” (NWO). The institute is internationally focused and renowned for its high quality research, a substantial fraction of which is carried out in the framework of (inter)national programs. Over 160 researchers conduct pioneering research and share their acquired knowledge with society. More than 30 researchers are employed as professors at universities. The institute has generated twenty-one spin-off companies. CWI maintains excellent relations with industry and the academic world, both in the Netherlands as well as abroad. After their research careers at CWI, an increasing number of young staff members find employment in these sectors or in spin-off companies that are based on research results from CWI. Of course, library and computing facilities are first-rate. CWI's non-scientific services to its personnel include career planning, training and courses, assistance in finding housing, and tailor-made solutions to problems that may occasionally arise. TERMS AND CONDITIONS The terms of employment are in accordance with the Dutch Collective Labour Agreement for Research Institutes ("CAO-onderzoeksinstellingen"). Moreover CWI offers attractive working conditions, including flexible scheduling and help with housing for expat employees. The gross monthly salary, for an employee on a full time basis, is €2,037 during the first year and increases to €2,610 over a four year period. CWI also offers excellent and flexible terms of employment, including an employee pension fund. INFORMATION Additional information can be obtained from: dr. Stratos Idreos, email stratos.idreos(a)cwi.nl, telephone +31(0)20 592 4169. or dr. Stefan Manegold, email stefan.manegold(a)cwi.nl, telephone +31(0)20 592 4212, or prof. dr. Martin Kersten, email martin.kersten(a)cwi.nl, telephone +31(0)20 592 4066. APPLICATION Please send your application before 31 July 2011 to: pd(a)cwi.nl. Applications should include a detailed CV, a motivation letter, a list of your M.SC. courses and grades, a copy of your master thesis, and preferably a list of publications.

1 0

[Monetdb-developers] PhD position at CWI Database Architecture group
by Stefan Manegold 14 Jun '11

14 Jun '11

CWI is an internationally renowned research institute in mathematics and computer science, located in Amsterdam, The Netherlands. The focus is on fundamental research problems, inspired by societal needs. Research is carried out in 18 research groups and a substantial fraction is carried out in the framework of (inter)national programs. VACANCY CWI has a vacancy for a talented PhD candidate in the DB Architectures group (http://www.cwi.nl/en/research-groups/Database-Architectures) DISTRIBUTED DATABASE ARCHITECTURES FOR SCIENCE APPLICATIONS The Database Architectures group is one of the world-wide leading groups in system-oriented database research, e.g., it received the 10-year best paper award at VLDB 2009 for their pioneering research on hardware-aware database technology and the column-store model. The group has a unique methodology that combines fundamental database research with active participation of all group members in database engineering, centered around the open-source analytical database system MonetDB (http://www.monetdb.org/) You will be expected to write both papers and contribute to this code base. The focus of the group is on hard core data management and for this position specifically on database architectures to support large scale scientific applications in the context of the SciLens project (http://www.scilens.org) The goal is to leverage the potential power of a 330 node dedicated database machine to improve science in, e.g., astronomy and seismology. (http://www.scilens.org/content/platform) CWI maintains excellent relations with industry and the academic world, both in the Netherlands as well as abroad. An internship at one of the partnering database research labs world-wide is considered part of the PhD track. After their research careers at CWI, many staff members find employment in these sectors or in spin-off companies based on research results from CWI. Of course, library and computing facilities are first-class. CWI's non-scientific services to its personnel include career planning, training and courses, assistance in finding housing, and tailor-made solutions to problems that may occasionally arise. REQUIREMENTS The PhD candidates must have a master degree in computer science, (specialization distributed systems, database, software engineering) mathematics, or related fields. Preferable qualifications for candidates include proven research talent, system programming skills (C/C++), practical experience with database systems, and software development in a team. One should bring good academic writing and presentation skills. Candidates are expected to have an excellent command of English. TERMS AND CONDITIONS The terms of employment are in accordance with the Dutch "CAO-onderzoeksinstellingen". Moreover, CWI offers attractive working conditions, including flexibility and help with housing for foreigners. The gross monthly salary (on a full time basis) will be â¬ 2,037 during the first year and will reach â¬ 2,610 during the fourth year. INFORMATION Additional information can be obtained from prof.dr. Martin Kersten, email martin.kersten(a)cwi.nl, telephone +31(0)20 592 4066, or dr. Stefan Manegold, email stefan.manegold(a)cwi.nl, telephone +31(0)20 592 4212. -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] Basics about MonetDB
by 叶亮 01 Jun '11

01 Jun '11

Hi, I'm a new comer with some questions about basic conceptions of MonetDB. 1. The executor I've ever found the executor of postgreSQL at src\backend\executor, but cannot make sure where it is like that in MonetDB src. MonetDB\MonetDB\src\gdk or MonetDB5-server\MonetDB5\src\mal ? 3. MAL MAL (MonetDB Assembly Language) is a script tool only for MonetDB or a common script language that can be applied to other softwares. Why it is a must in MonetDB ? 4. .mx files In MonetDB, .mx files are compiled to make .h and .c files generated.(in this way ?) Why is this routines ? (not just coding in .h and .c files directly) Yel ---- Best Regards richardroky(a)gmail.com

8 10

[Monetdb-developers] Announcement: New Apr2011-SP1 bugfix release
by Fabian Groffen 31 May '11

31 May '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011-SP1 bugfix release of the MonetDB suite of programs. At the time of this release, we also officially changed to our new website at http://www.monetdb.org/. This new website is our central source of information, and comes with our new logo and style. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>.

1 0

[Monetdb-developers] Django adapter for MonetDB 11.3.1
by Marco Quartulli 30 May '11

30 May '11

Dear all, an adapter to the Django web development framework for MonetDB 11.3.1 is available at https://github.com/marcoq/monetdb-python The code is functional and already running an experimental image retrieval site, yet full tests will need to be passed in order to consider inclusion in the Django tree as a proper backend. A merger into the source tree at https://github.com/vikram/monetdb-python has been requested. Marco

2 1

[Monetdb-developers] Qualifying and fixing issues found by the test suite
by Gregory Burd 12 May '11

12 May '11

Hello all, I'm new to MonetDB, but I'm not new to databases or coding. I've been building the Apr2011 branch on my home system, a Fedora release 14 (Laughlin) (8x Intel(R) Xeon(R) CPU X3440 @ 2.53GHz, Memory 8187MB) using clang (clang version 2.8 (branches/release_28) Target: x86_64-redhat-linux-gnu Thread model: posix) and having great success - I am thrilled to be able to work with such an amazing piece of software that has such an elegant and well thought out engineering design and structured methodology (nice work!). Everything builds and runs as expected. However when I run "make check" there are a number of things that catch my eye and I want to know from the experts how you'd like me to help with resolving these potential issues. First, here's how I'm building things: export JAVA_HOME=/usr/java/jdk1.5.0_22 export PATH=${JAVA_HOME}/bin:$PATH export CC=clang ./configure --enable-testing --enable-developer --enable-debug --enable-assert --enable-instrument --enable-merocontrol --enable-strict --enable-warning --enable-dependency-tracking --enable-bits=64 --enable-jdbc --enable-odbc make -k --load-average=90 --jobs=`cat /proc/cpuinfo | grep processor | wc -l` && make check Failures include "major differences" found in the following: tst830.mal in mal vacuum.mal in modules/kernel attach00.mal in modules/mal common.mal in optimizer perl_dbi.MAL.sh in perl/dbi The attach differences I'm seeing don't seem (at first glance) to match those of http://bugs.monetdb.org/show_bug.cgi?id=2751 My general question is, should I report these as bugs? What kind of information from the test run would you like me to add to the bug record? How would I find out if these are already reported (I've tried a few searches already)? I'm happy to dig in and start working on these issues, contributing patches where/when I can. I hope to follow whatever established patterns there are for developing/fixing MonetDB. Also, I see a lot of output like "(attach00.test.err.FILTERED) significantly" and "(pithole.test.out.FILTERED) significantly" what does this mean? Are there tests being skipped? Is there some issue/error that's being filtered out (removing noise so as to better see the signal)? Is there a way to say, run all the tests including those that are expected to fail? Another, semi on-topic, question. I happen to use emacs, has anyone taken the time to write up a .mx mode so that it's a bit easier to develop against the .mx source files? Any debugging hints/tips/tricks hidden on the web anywhere? best, -greg

1 0

[Monetdb-developers] JDBC 4 effort
by Christophe Galerne 01 May '11

01 May '11

Hi, Is there an effort to make the JDBC driver JDBC 4 compliant ? If this is the case, is there a way to participate ? Cdlt, Christophe

2 1

[Monetdb-developers] Announcement: New Apr2011 feature release of MonetDB suite
by Sjoerd Mullender 28 Apr '11

28 Apr '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Apr2011 feature release of the MonetDB suite of programs. As announced one month ago to the day (March 28, 2011), this release marks the start of a new era: development of MonetDB4 and MonetDB/XQuery has ceased. Development of the Pathfinder compiler continues unabated at the University of Tübingen [1], but there is no longer a link being maintained with MonetDB. For details on this release, please see the release notes at <http://monetdb.cwi.nl/Development/Releases/Apr2011/>. [1] http://www.pathfinder-xquery.org/ -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New Mar2011 feature release of MonetDB suite
by Sjoerd Mullender 28 Mar '11

28 Mar '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Mar2011 feature release of the MonetDB suite of programs. This release marks the last planned release of MonetDB4 and Pathfinder, and hence the last planned release of MonetDB/XQuery. Development of the Pathfinder compiler continues at the University of Tübingen. The next release, currently planned for April 2011, should be a smooth upgrade from this release, even if MonetDB/XQuery is installed (which will of course then not be upgraded). This release also marks the first release of the new way of building the MonetDB suite. For details on both these issues, and on the other features and bug fixes of this release, please see the release notes at <http://monetdb.cwi.nl/Development/Releases/Mar2011/>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] snappy library for BAT compression?
by Roberto Cornacchia 25 Mar '11

25 Mar '11

>From opt_compression.mx: @f opt_compression @- The compressed database Column stores mostly exploit some form of compression to reduce the footprint on disk and to improve disk/memory/cache bandwidth at the expense of cpu/memory cycles. A naive approach is to take a (*preferrably lightweight*) compression algorithm and deploy it against the units of access, e.g. disk/buffers. In MonetDB our granularity is a BAT, which therefore forms the basis for (de)compressed access. I don't know whether this is still an actively developed idea. If it is, perhaps this can help: http://code.google.com/p/snappy/ Roberto

1 0

[Monetdb-developers] New Apr2011 stable branch created
by Sjoerd Mullender 22 Mar '11

22 Mar '11

I have created new stable branches for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Apr2011. So in each of your stable checkout directories you can do hg pull hg update -r to update to this new branch. Note that there is still the Mar2011 branch as well. -- Sjoerd Mullender

1 0

[Monetdb-developers] Search MonetDB Source - character processing at light speed :-)
by Arjen P. de Vries 21 Mar '11

21 Mar '11

Dear MonetDB enthusiasts, Feel free to use the suffix array search demo of the MonetDB source tree where you see fit: http://devel.spinque.com/SearchMonetDBSource/ The index is refreshed every night! You will interactively view all the exact matches of the string you type (with their count); while you type it - like Google Instant, but then per character instead of word. (Nice examples for a first try: allready, is build, != null, ...) For more complex needs, an extended syntax is supported. Here, all individual operands have to be delimited by single quotes. (Infix) operators supported are < and > for before and after, respectively, and ~ for "near". After the second operand, specify the window size in characters. Examples: '#define'<('max','min')30 finds occurrences of #define before either max or min, in a 30 character window. 'if'<'== -1'80~'&&'50 Finds 'if' appearing before '== -1' in an 80 char window, together with '&&' in a char window. We are open to suggestions (send to info(a)spinque.com to reach all of us), and keen to hear about your successful (and even the unsuccessful) uses of the demo. Many greetings, The Spinque team Arjen, Roberto and Wouter -- ==================================================================== CWI, room L333 Centre for Mathematics and Computer Science Science Park 123 Email: Arjen.de.Vries(a)cwi.nl 1098 XG Amsterdam tel: +31-(0)20-5924306 The Netherlands fax: +31-(0)20-5924199 ===================== http://www.cwi.nl/~arjen/ ====================

3 2

[Monetdb-developers] draft message
by Martin Kersten 01 Mar '11

01 Mar '11

Dear MonetDB users and developers, The preparations for the March 2011 release are on target. This release will feature a much simplified installation and compilation process using a unified bootstrap/configure/make/install. Ofcourse, installers and RPMs are also provided. Furthermore, the March 2011 release will be the freeze of the MonetDB XQuery source tree. It will remain downloadable from the website, but no active development or maintenance is foreseen. In April 2011 a MonetDB/SQL feature release will mark the split of the two code bases. From that point onward, we will primarily be concerned with SQL and its enhancements. Saying goodbye to the very old mserver4 code base, also triggered a major code renovation activity. The MonetDB/SQL code base will be cleaned from legacy code, 1-time experimental code, and all that is not strictly needed for the intended functionality. We expect that this cleanup will simplify maintenance in the future. The results of the renovation step will become visible in the fall of 2011. We plan for a smooth transition step for existing SQL environments. However, under the hood, we will change the MAL and kernel library interfaces significantly. To assess the consequences in the community we would like to learn about projects that directly address the MAL or GDK layer. Please send us an email at info(a)monetdb.org with the project highlights. regards, Martin Kersten

1 0

[Monetdb-developers] Concerning headless branch
by dsuh 15 Feb '11

15 Feb '11

I noticed a new headless branch that aims to replace BATs with COLs. Could you give me more information on COLs? Is what you are trying to achieve faster queries, less memory usage, or other advantages? Assuming that this is the new direction for MonetDB, do you expect that it will be merged with the Hg default tip for the next release after March 2011? Thanks. David Suh

2 1

[Monetdb-developers] libxml linking error during compilation, Mar2011 (Mac 10.6.6)
by Babis Nikolaou 14 Feb '11

14 Feb '11

Hi, do you have any idea about the following error: *** Warning: Linking the shared library lib_batxml.la against the loadable module *** lib_xml.so is not portable! *** Warning: lib lib_xml.so is a module, not a shared library *** And there doesn't seem to be a static archive available *** The link will probably fail, sorry Attached are the libtool's actions concerning the previous error. Furthermore, if it is relevant to my question, i provide the following information: MonetDB_Mar2011$ find . -name 'lib_xml*' ./monetdb5/extras/xml/.libs/lib_xml.la ./monetdb5/extras/xml/.libs/lib_xml.lai ./monetdb5/extras/xml/.libs/lib_xml.so ./monetdb5/extras/xml/.libs/lib_xml.so.dSYM ./monetdb5/extras/xml/.libs/lib_xml.so.dSYM/Contents/Resources/DWARF/lib_xml.so ./monetdb5/extras/xml/.libs/lib_xml_la-xml.o ./monetdb5/extras/xml/lib_xml.la ./monetdb5/extras/xml/lib_xml_la-xml.lo And last, i can assure you that there is a static library in my system for libxml (/usr/local/Cellar/libxml2/2.7.7/lib/libxml2.a) and the directory containing it has been included in the PKG_CONFIG_PATH explicitly (you can check this by inspecting the attached file). Thanks, Babis

3 3

[Monetdb-developers] Installation troubles Mac OSX 10.6.6
by Babis Nikolaou 11 Feb '11

11 Feb '11

Hi, what is going about this issue on Macs? Is there any progress? http://sourceforge.net/mailarchive/message.php?msg_id=26996821 I have managed to overcome all but the last problem concerning the AC_MSG_ERROR. The solution was to copy all .m4 files provided from the gettext package into the MonetDB build directory. Thanks, Babis

3 4

[Monetdb-developers] Installation troubles Mac OSX 10.6.6
by Jan Flokstra 02 Feb '11

02 Feb '11

Hello, I am trying to install the newest monetdb system with the single bootstrap/configure on my Mac. This does not work as good as it used to be. First when I do ./bootstrap I get the message that the libtoolize cannot be found. This is not installed on Mac OS, the default here is glibtoolize (version = 2.2.4). When using a package managers like fink libtoolize may be already installed in /sw because other packages need it. My solution was to temporary change the bootstrap file. The next problem is that it cannot find the libiconv package. I have libiconv installed in the monetdb prefix but i get the message: configure.ac:2255: error: possibly undefined macro: AM_ICONV When I remove this macro from bootstrap I get the message: configure.ac:152: error: possibly undefined macro: AC_MSG_ERROR This is a hard one because this is a standard macro, What could be wrong here? Jan Flokstra.

2 4

[Monetdb-developers] New Mar2011 stable branch created
by Sjoerd Mullender 31 Jan '11

31 Jan '11

I have created new stable branches for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch id Mar2011. So in each of your stable checkout directories you can do hg pull hg update -rMar2011 to update to this new branch. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] Packages for OSX (homebrew)
by Holger Pirk 18 Jan '11

18 Jan '11

Hi Gabriele, I actually have a small script that does that. I will talk to fabian about integrating that into the build-process. However, I don't really see it as urgent for your case. I feel that the main point of most of these package managers (darwinports, fink, homebrew, …) is that you get binaries that are optimized for your hardware. The prebuild binary obviously can't provide that anyway, so the package that is build from the sources by the package manager is way more important. kind regards holger On Jan 18, 2011, at 10:54 AM, Gabriele Modena wrote: > On Mon, Jan 17, 2011 at 10:54 PM, Holger Pirk <holger(a)airbug.de> wrote: > > Hi Holger, > >> One thing did strike me as odd in your code, though: When installing from the binary archive the absolute file names of the library dependencies are hard-coded into the mach-o binaries. This would break the installation unless installed into the standard prefix (/usr/local). > > I was aware of this problem and did not not know install_name_tool. I > will look into a workaround tonight. > > Thanks for the hint! > > -- > Gabriele > > ------------------------------------------------------------------------------ > Protect Your Site and Customers from Malware Attacks > Learn about various malware tactics and how to avoid them. Understand > malware threats, the impact they can have on your business, and how you > can protect your company and customers by using code signing. > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers

2 1

[Monetdb-developers] Packages for OSX (homebrew)
by Gabriele Modena 18 Jan '11

18 Jan '11

Hi all, over the weekend I packaged MonetDB-Oct2010-SP1 for homebrew, an increasingly popular git-based package manager for OSX. For more info about homebrew please see: http://mxcl.github.com/homebrew/ The package can be found at https://github.com/gmodena/homebrew/blob/master/Library/Formula/monetdb.rb I will be doing some more tests, tweaks and clean ups during the week, if everything works well I'd like to try to get the package included in the official repository. Please let me know if you have any comment or feedback. = Synopsis = # Install x86/x86_64 binaries $ brew install monetdb # Build from superball source $ brew install --superball monetdb = Features = - possibility to install either from binaries (x86/x86_64) or building from source (superball). Binaries are picked given the detected architecture. - plist file for (optionally) starting merovingian at boot/login = TODO = - add the possibility to build&install from the hg repository Cheers -- Gabriele

3 4

Re: [Monetdb-developers] MonetDB: default - Top-level bootstrap/configure/make.
by Stefan Manegold 07 Jan '11

07 Jan '11

Dear all, since I have not yet managed to update the TestTools to cope with Sjoerd's changes, i.e., the new (single) top-level bootstrap/configure/make, I will suspend "Current" ("default") testing as of tonight, until I manage to update (or reimplement) the TestTools accordingly. Please accept my aplologies for any inconveniences that this might cause. Stefan On Fri, Jan 07, 2011 at 01:41:44PM +0100, Sjoerd Mullender wrote: > Changeset: 09ddaf9782c1 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=09ddaf9782c1 > Modified Files: [1219] > Branch: default > Log Message: > > Top-level bootstrap/configure/make. > > This huge changeset is to turn the various "sort-of" independent > subpackages of MonetDB into one coherent package. The principle idea > is that we now have a single top-level configure and make instead of > having to run configure and make many times to build everything. > > >From now on, the way to build MonetDB is to run the command > "./bootstrap" in the top-level directory, and then to run > ".../configure" and "make" in the build directory (which can be the > same as the source, but doesn't have to be). On Windows, the way to > build is to run "nmake" in the NT subdirectory or to run "nmake /f > ..\NT\Makefile" in an empty directory underneath the top level. > > The subpackages affected are: MonetDB, clients, MonetDB4, MonetDB5, > sql, pathfinder, geom, testing, java, and python. Not affected is > template (although it likely won't currently work) since it is meant > as a template for external (to the MonetDB suite) extensions. > > Note that the python subpackage is included in the list: there is now > support to run the Python setup.py program with autogen. > > The top-level configure has a number of options to enable and disable > the building of subpackages: > --enable-monetdb4 enable support for MonetDB4 (default=auto) > --enable-monetdb5 enable support for MonetDB5 (default=auto) > --enable-sql enable support for MonetDB/SQL (default=auto) > --enable-geom enable support for geom module (default=auto) > --enable-pathfinder enable support for MonetDB/XQuery (default=auto) > --enable-pftijah enable support for TIJAH (default=auto) > --enable-probxml enable support for PROBXML (default=auto) > --enable-odbc compile the MonetDB ODBC driver (default=auto) > --enable-testing enable support for testing (default=auto) > --enable-console enables direct console on the server (involves > security risks) > --enable-java (try to) build Java components > --enable-jdbc build the MonetDB JDBC driver > --enable-xrpcwrapper build the MonetDB XRPC wrapper > --enable-merocontrol build the Merovingian control driver > > The options to specify the presence and location of programs and > libraries that are used to build the subpackages were of course also > integrated into the top-level configure, as were the following > options: > --with-default-backend=BACKEND > select a default back-end > (milprint_summer/algebra/sql) for Pathfinder > (default=algebra) > --with-password-backend=HASHALG > password hash algorithm, one of MD5, SHA1, > RIPEMD160, SHA224, SHA256, SHA384, SHA512, defaults > to SHA512 > > Note that the Pathfinder configuration without MonetDB4 is still possible: > .../configure --disable-monetdb4 --disable-monetdb5 --enable-pathfinder ... > > Other changes include the following: > - One top-level MonetDB.spec file so that all RPMs can be built in one > go; > - One top-level debian subdirectory with Debian/Ubuntu configuration; > - Removed all *-config and *-config.bat files: there use is superseded > by the single top-level configure and various *.pc (pkgconfig) files; > - All paths in Makefile.ag files to other subpackages are now relative; > - .mx files in the mel package have been unpacked to their constituent > .cc (etc.) files: adding back support for C++ inside autogen was too > hard otherwise; > - Tar balls don't include generated files (apart from configure): the > build process, also for tar ball distributions, includes building > and using Mx, but also now requires bison/yacc and flex/lex (there > have been problems in the past with our distributing generated bison > output); > - mel is only built when MonetDB4 is enabled, burg is only built when > pathfinder is enabled; > - Some symbols in burg were replaced to avoid name clashes on Windows; > - Some files (monetdb.ico, banner.bmp, website.html, license.rtf) were > moved to buildtools/conf; > - Use the currently approved way (by autoconf) of checking for alloca > and alloca.h; > - Packaged vault.sql into vault.mx since autogen/Mx get confused > otherwise; > - Set the version number globally to 11.0.0 (we need a single version > number for binary packages, and it must be higher than the highest > old version number). > > -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] changes to building MonetDB
by Sjoerd Mullender 05 Jan '11

05 Jan '11

This is an important announcement for everyone who wants to build MonetDB from sources (tar balls and Mercurial). This announcement is only relevant to the "default" branch. Pretty soon I will commit and push a big change to the way MonetDB is built. This message is a heads-up for that change and contains information on how to build MonetDB in the (near) future. Currently, in order to build the MonetDB suite, you need to do the following: * build and install buildtools (only if building from Mercurial sources); * build and install MonetDB; * build and install clients; * build and install MonetDB4; * build and install MonetDB5; * build and install sql; * build and install pathfinder; * build and install geom; * build and install java; * build and install python; * build and install testing. Some of the above are optional, and some variation in the order is possible. In all cases (except python), building and installing consists of running: * ./bootstrap * .../configure ... * make * make install This whole process is going to change. Very soon you will need to run the commands * ./bootstrap (not for tar balls) * .../configure ... * make * make install only once in the top-level directory of the MonetDB suite (well, ./bootstrap has to run there, the other commands can run either there or in a separate build directory (recommended)). For now, there are no changes to the way the Python module is built. The advantage of this change is clear: you only need to run these commands once instead of 9 times. This means that both bootstrap and configure will be faster, even though the single instance needs to do a bit more work than any single instance in the old situation (but there were 9 of those). Another advantage is that things like * make dist (create tar ball) * make rpm (create binary RPM) * make srpm (create source RPM) also need to be run only once for the complete suite. We no longer have a separate Super Source Tar Ball which was just a tar ball containing all the separate source tar balls. Note that in the old situation, there were a whole bunch of configuration options in the Pathfinder configure. These options have been moved to the top-level configure. It used to be possible to only configure and build Pathfinder. That is now supported by using options on the top-level configure disabling building of MonetDB4 and MonetDB5 subpackages: .../configure --disable-monetdb4 --disable-monetdb5 [pathfinder-specific config options] Attached is the check-in message that I plan to use (modulo last minute changes). -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New Jun2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 21 Dec '10

21 Dec '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010-SP1 bug fix release of the MonetDB suite of programs. The most important and visible change is a security fix in the SQL COPY INTO command. This is detailed in the release notes. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] updating the default branch.
by Romulo Goncalves 25 Nov '10

25 Nov '10

Hello, I just updated my default branch and I restarted my mserver, i.e., it will use a old dbfarm. I get the following complain: !MALException:user.main[2497]:address of pattern vault.prelude missing I hope you will not ask me to reload my databases :) Romulo

4 11

[Monetdb-developers] Announcement: New Oct2010 feature release of MonetDB suite
by Sjoerd Mullender 15 Nov '10

15 Nov '10

It is not October anymore, but nevertheless, the MonetDB team at CWI/MonetDB BV is pleased to announce the Oct2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Oct2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] MonetDB: Oct2010 - approved output
by Stefan Manegold 30 Oct '10

30 Oct '10

Niels, is it by accedent or by design that with mitosis & mergetable, mat.slice() is not longer used with LIMIT ? If by design, we'd need to change (or even drop) test mat.slice_limit1.Bug-2645, that fails single-threaded since you below checkin; cf., http://monetdb.cwi.nl/testing/projects/monetdb/Candidate/sql/.mTests103/GNU… If by accident, we'd need to investigate whether this is indeed OK ... Stefan On Wed, Sep 15, 2010 at 03:18:54PM +0200, Niels Nes wrote: > Changeset: d3918481778e for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d3918481778e > Modified Files: > sql/src/test/BugTracker-2010/Tests/LIMIT_OFFSET_big-endian.Bug-2622.stable.out > sql/src/test/BugTracker-2010/Tests/copy-into-too-long-string.Bug-2358.sql > sql/src/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out > sql/src/test/BugTracker-2010/Tests/with_row_number_crash.Bug-2631.stable.out > Branch: Oct2010 > Log Message: > > approved output > [...] > diff -r b3f860694c8a -r d3918481778e sql/src/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out > --- a/sql/src/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out Wed Sep 15 15:12:37 2010 +0200 > +++ b/sql/src/test/BugTracker-2010/Tests/mat.slice_limit1.Bug-2645.stable.out Wed Sep 15 15:18:49 2010 +0200 > @@ -112,7 +112,7 @@ > % L6 # name > % wrd # type > % 1 # length > -[ 1 ] > +[ 0 ] > #TRACE select cast(x as string)||'-bla-'||cast(y as string) from slice_test limit 1; > % sys. # table_name > % concat_concat_x # name > @@ -127,7 +127,7 @@ > % L6 # name > % wrd # type > % 1 # length > -[ 1 ] > +[ 0 ] > #drop function GetTrace; > #drop table slice_test; > [...] -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0

[Monetdb-developers] help troubleshoot: cannot connect to MonetDB
by Tripathi, Jaya 25 Oct '10

25 Oct '10

C:\Proximity\bin>gui.bat localhost:30000 0 [main] WARN kdl.prox.util.Util - * log config file not found (using defaults) : C:\Proximity\bin\prox.lcf' Exception in thread "main" Unable to connect: Connection refused: connect at kdl.prox.monet.BlockedModeStream.<init>(BlockedModeStream.java:162) at kdl.prox.monet.Connection.open(Connection.java:401) at kdl.prox.db.DB.open(DB.java:935) at kdl.prox.db.DB.open(DB.java:914) at kdl.prox.app.GUI2App.main(GUI2App.java:238)

1 0

[Monetdb-developers] MonetDB deadlocks on clients, now what?
by Stefan de Konink 19 Oct '10

19 Oct '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Lately MonetDB starts to deadlock on the clients. Bugs were reported, typically I have connected ~8 clients connected at the same time. http://bugs.monetdb.org/show_bug.cgi?id=2670 Do other people observe this? Is there a good way to debug where it comes from, or show how it hangs? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAky9jXwACgkQYH1+F2Rqwn0DogCglDldSCPcnkRxFIGbtTG/P6ER 3XgAn3uMX45p6UmJrK0aNSFwo6SglSPz =ldsT -----END PGP SIGNATURE-----

1 0

[Monetdb-developers] test missing on pkg-config in clients configure
by Jan Rittinger 10 Oct '10

10 Oct '10

Hi all, I got the following error message due to a missing pkg-config installation—Perhaps there could be a better error message: /home/rittinge/MonetDB.hg/clients/configure: line 2894: syntax error near unexpected token `monetdbtools,' /home/rittinge/MonetDB.hg/clients/configure: line 2894: `PKG_CHECK_MODULES(monetdbtools, monetdb-stream >= $req_monetdb_ver' -- Jan Rittinger Lehrstuhl Datenbanken und Informationssysteme Wilhelm-Schickard-Institut für Informatik Eberhard-Karls-Universität Tübingen http://www-db.informatik.uni-tuebingen.de/team/rittinger

2 5

[Monetdb-developers] write error
by Mark Bucciarelli 07 Oct '10

07 Oct '10

Hi, I'm getting a "write error" when running a fairly large and complex query (five tables and subselect in where). I run $ mclient -l sql -d <db> -u <usr> < t.sql password: ... % 6, 26, 1, 103, 229, 5 # length write error $ echo $? 1 I've run the exact query on SQLite, so I'm pretty sure the SQL is good. This is Feb2010. I'm thinking about upgrading, but I'm close to a release so I'd prefer not to. Does this description remind anyone of a commit to Jun2010 that may have fixed such an issue? Trying to boil the (complex) SQL down to a small reproduceable test case will take a lot of time. Is there another approach I can use to debug that would be faster? There is nothing in the merovingian log. Thanks, m

3 2

[Monetdb-developers] Obsolete reference to buildtools\conf\winrules.msc
by Sander Kruseman 06 Oct '10

06 Oct '10

Dear MonetDB team, I am trying to build a MonetDB client on Windows, following the instructions in "1.24 Building MonetDB On Windows". In every package's folder named "NT" is a file named "rules.msc" which references: \..\..\buildtools\conf\winrules.msc. But none of the (nightly or release) tarballs contains the buildtools folder. My question: are the tarballs not expected to be built from, and should I choose a different route, or is the reference obsolete and can I fix this myself until the distro is fixed? Background: Actually all I need is the file mapilib.lib, in order to be able to use the mapi programming interface using mapilib.dll. Since this file is not provided, I hope to create it myself by building the clients component (on Win64). cheers, Sander Kruseman

3 3

[Monetdb-developers] Why the Speed of Mapi Lib is so lower?
by peter qin 06 Oct '10

06 Oct '10

Dear all, Yesterday,I had done a test about the speed of MonetDB.I used the example code make a test.the red part is my adding code. #include <mapilib/Mapi.h> #include <stdio.h> #include <stdlib.h> void die(Mapi dbh, MapiHdl hdl) { if (hdl != NULL) { mapi_explain_query(hdl, stderr); do { if (mapi_result_error(hdl) != NULL) mapi_explain_result(hdl, stderr); } while (mapi_next_result(hdl) == 1); mapi_close_handle(hdl); mapi_destroy(dbh); } else if (dbh != NULL) { mapi_explain(dbh, stderr); mapi_destroy(dbh); } else { fprintf(stderr, "command failed\n"); } exit(-1); } MapiHdl query(Mapi dbh, char *q) { MapiHdl ret = NULL; if ((ret = mapi_query(dbh, q)) == NULL || mapi_error(dbh) != MOK) die(dbh, ret); return(ret); } void update(Mapi dbh, char *q) { MapiHdl ret = query(dbh, q); if (mapi_close_handle(ret) != MOK) die(dbh, ret); } int main(int argc, char *argv[]) { Mapi dbh; MapiHdl hdl = NULL; char *name; char *age; dbh = mapi_connect("localhost", 50000, "monetdb", "monetdb", "sql", "demo"); if (mapi_error(dbh)) die(dbh, hdl); update(dbh, "CREATE TABLE emp (name VARCHAR(20), age INT)"); for(int i = 0;i<10000;i++) { update(dbh, "INSERT INTO emp VALUES ('John', 23)"); } //why this cast more then serveral minutes .can you help me? hdl = query(dbh, "SELECT * FROM emp"); while (mapi_fetch_row(hdl)) { name = mapi_fetch_field(hdl, 0); age = mapi_fetch_field(hdl, 1); printf("%s is %s\n", name, age); } mapi_close_handle(hdl); mapi_destroy(dbh); return(0); } Peter Qin peterqin1980(a)gmail.com

3 2

Re: [Monetdb-developers] MonetDB: Oct2010 - Approve -5 option.
by Stefan Manegold 04 Oct '10

04 Oct '10

Thanks, Sjoerd! ... as question mainly towards the pathfinder developers in Tübingen: -5, --m5-sql: generate MIL code ^^ ^^^ ^^^ Does the "-5" option generate SQL, MIL, or MAL code? Looking at the actual output of, e.g., `echo '1+2' | pf -5`, it does not look like either of the above to me, but rather like dot output... Thanks in advance for clearifying! Stefan On Mon, Oct 04, 2010 at 09:56:19AM +0200, Sjoerd Mullender wrote: > Changeset: abe5327b4915 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=abe5327b4915 > Modified Files: > pathfinder/compiler/Tests/usageH.stable.out > Branch: Oct2010 > Log Message: > > Approve -5 option. > > > diffs (11 lines): > > diff -r 69feea05a494 -r abe5327b4915 pathfinder/compiler/Tests/usageH.stable.out > --- a/pathfinder/compiler/Tests/usageH.stable.out Sun Oct 03 21:41:14 2010 +0200 > +++ b/pathfinder/compiler/Tests/usageH.stable.out Mon Oct 04 09:40:25 2010 +0200 > @@ -18,6 +18,7 @@ > Reads from standard input if FILE is omitted. > > = -A, --mil: generate MIL code (default) > + -5, --m5-sql: generate MIL code > = -M, --mil_2004: generate MIL code (directly from XQuery Core) > = -S, --sql: generate SQL code > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] Large XML instances with MonetDB/XQuery 4.38.5
by Christian Grün 02 Oct '10

02 Oct '10

Hi everyone, is it still possible to shred large XML instances with MonetDB/XQuery, or do I need more memory for shredding? I was just trying to the 1GB instance of XMark with WinXP 32Bit and 2GB RAM, and this is what I got: MonetDB>shred_doc("1gb.xml", "xmark"); #GDKmmap(536870912) fails, try to free up space [memory in use=36143780,virtual memory in use=1015480320] #GDKmmap(536870912) result [mem=26016940,vm=1005387776] !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 536870912 for 06\632theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( follow murders prov shoes p resent eager delivered defy bend viol here marquis powers complexion pillar cran ts agate ), BUNappend fails !ERROR: CMDshred_url: operation failed. pf:add-doc(...) produces the same output; the XML document itself is well-formed. Thanks, Christian

2 6

[Monetdb-developers] How to commit into MonetDB repo.
by Alexandr Kulik 23 Sep '10

23 Sep '10

Hi All, I had implemented a few statistical function for MonetDB SQL. Standard deviation and variance for sample and population. I'm interesting is it possible to commit this code into repo, so everyone who need them could use it. This changes will partially cover bug # 2454. -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

3 3

Re: [Monetdb-developers] How to enable functions via sql?
by Stefan de Konink 23 Sep '10

23 Sep '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 23-09-10 20:54, Alexandr Kulik schreef: > I implement few functions (standard deviation sample and population, > variance sample ad population). Is it possible to commit them into > MonetDB repo? Again, use the monetdb-dev list. I guess your best shot is to log a bug for it at: <http://bugs.monetdb.org/> and mark it as enhancement. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkybq/kACgkQYH1+F2Rqwn1m/QCfXje9sE6hNYXhkp9IfUnGPo/n dw0An1d64ZiGZ07NCeJY+BgxaPVkyLLW =1hPu -----END PGP SIGNATURE-----

1 0

Re: [Monetdb-developers] How to enable functions via sql?
by Stefan de Konink 20 Sep '10

20 Sep '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 20-09-10 19:57, Alexandr Kulik schreef: > I find how could i do it. Now i fighting with another problem. Is it > possible from monetdb sql side create aggregation function that will > receive 2 arguments? Probably e-mailing the monetdb-dev list is a better place to ask such questions. But yes this must be possible. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkyXv3wACgkQYH1+F2Rqwn3Q3QCfQLHj8lz/osJxGu/nPL96a+0G YQwAoIWMLBQd4SpaQ3sjNia7kOHwDfy6 =8YsW -----END PGP SIGNATURE-----

1 0

[Monetdb-developers] About MonetDB query optimize
by xixi 17 Sep '10

17 Sep '10

Dear all, Hello,Recently I am intersting in query optimize, and have know that most database's query optimizing methods contain two ：1. Rule Base Optimizer; 2.Cost Base Optimizer. I also know that MonetDB has two-level optimizing ,one is relation algebra optimize and the other is MAL optimize when interprets a sql. I understand relation algebra optimize is no relationship with the column-oriented special query optimize.MAL optimizer may be some relationship with the column-oriented special query optimize. So I want to ask :MAL optimize is also contain Rule Base Optimizer and Cost Base Optimizer? which optimizers is the special query optimizer for the column-oriented and not for tuple-oriented database? Best regards!

1 0

Re: [Monetdb-developers] How to enable functions via sql?
by Alexandr Kulik 16 Sep '10

16 Sep '10

Stefan, I understand how to create function from sql that receive value as input param(s). But how could i write sql function that receive array of values as input? For example i want to enable stdev function at algebra module, is it possible to do from sql? Alex Op 06-09-10 18:13, Alexandr Kulik schreef: > I need to develop some extra statistical functions for MonetDB. I > spent about a week to understand how to use function via sql, but > still didn't have any succes results. May be someone could point me to > appropriate documentation or source revision, where i could look on > it? You might be interested in these two files it shows how to add a module to SQL written in the mx language: http://kinkrsoftware.nl/contrib/monetdb/sphinx.mx http://kinkrsoftware.nl/contrib/monetdb/sphinx.patch After that you will still have to 'create function', but that is relatively simple. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkyFFBUACgkQYH1+F2Rqwn1lRwCfe02Bjv3YozRMu3jaUug1bMgD lXYAnAn3ktCoAEZGq7CEr0aCAfKRCWNs =FNsy -----END PGP SIGNATURE----- -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

1 0

Re: [Monetdb-developers] MonetDB: Oct2010 - When you have MAL plans with 500k lines of co...
by Romulo Goncalves 15 Sep '10

15 Sep '10

Hello, I fixed the alloca problem, but I am wondering if the newStack function is correct. On 15/09/10 15:56, Romulo Goncalves wrote: > Changeset: 2727702e548b for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2727702e548b > Modified Files: > MonetDB5/src/mal/mal_interpreter.mx > MonetDB5/src/mal/mal_stack.mx > Branch: Oct2010 > Log Message: > > When you have MAL plans with 500k lines of code mserver crashes at the memset after newStack. > This function is only called in one place and followed by a memset. Therefore, the alloca + memset > can be replaced by a zalloc. > > The structure is free in the end of the function call before the return. > > > diffs (32 lines): > ... > #define stackSize(CNT) (sizeof(ValRecord)*(CNT) + sizeof(MalStack)) > -#define newStack(S,CNT) S= (MalStkPtr) alloca(stackSize(CNT));\ > +#define newStack(S,CNT) S= (MalStkPtr) GDKzalloc(stackSize(CNT));\ > (S)->stksize=CNT; I do not know the context, but shouldn't the stack size be stackSize(CNT) instead of CNT? Romulo > > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

1 0

Re: [Monetdb-developers] Restart mclient
by Romulo Antonio Goncalves 07 Sep '10

07 Sep '10

Indeed, The problem is gone. Thanks a lot. Regards, Romulo ----- Original Message ----- From: "Sjoerd Mullender" <sjoerd(a)acm.org> To: monetdb-developers(a)lists.sourceforge.net Sent: Tuesday, September 7, 2010 3:22:47 PM Subject: Re: [Monetdb-developers] Restart mclient The culprit seems to be changeset f2ceaaf24796: changeset: 37471:f2ceaaf24796 parent: 37469:3733beae1496 user: Fabian Groffen <fabian(a)cwi.nl> date: Fri Sep 03 15:27:26 2010 +0200 files: MonetDB5/src/mal/mal_session.mx description: bugzie says that bug #1939 is fixed, so disable the guard On 2010-09-07 16:02, Romulo Antonio Goncalves wrote: > Hello, > > I updated my default branch and I do the following: > > mclient -lmal >> \q > mclient -lmal >> \q > > After the second exit I get a crash in the mserver: >> [New Thread 0x7fffdf7ac710 (LWP 19441)] > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fffdf7ac710 (LWP 19441)] > 0x00007ffff7cd7198 in MALexitClient (c=0x605b38) at > /ufs/goncalve/MonetDB/current/MonetDB5/src/mal/mal_session.mx:522 522 > if (c->glb && c->curprg->def->errors == 0) > (gdb) p c->curprg->def > $1 = (struct MALBLK *) 0xdbdbdbdbdbdbdbdb > > Similar problem if you do > mclient -lsql >> \q > mclient -lsql > > The mserver crashes due to the same problem: > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fffdf7ac710 (LWP 19561)] > 0x00007fffdfe22a60 in SQLparser (c=0x605b38) at > /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_scenario.mx:1155 > 1155 oldvtop = c->curprg->def->vtop; > (gdb) p c->curprg > $5 = (Symbol) 0x7fffd8112198 > (gdb) p *c->curprg > $6 = {peer = 0xdbdbdbdbdbdbdbdb, skip = 0xdbdbdbdbdbdbdbdb, > name = 0xdbdbdbdbdbdbdbdb <Address 0xdbdbdbdbdbdbdbdb out of > bounds>, kind = -606348325, def = 0xdbdbdbdbdbdbdbdb} > (gdb) > > > Were there any changes in the client structure or in the > initialization of the of the client threads? > > Regards, > Romulo > > PS: It does not happen in the Oct2010 branch. > > ------------------------------------------------------------------------------ > This SF.net Dev2Dev email is sponsored by: > > Show off your parallel programming skills. > Enter the Intel(R) Threading Challenge 2010. > http://p.sf.net/sfu/intel-thread-sfd > _______________________________________________ Monetdb-developers > mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- Sjoerd Mullender ------------------------------------------------------------------------------ This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers

1 0

[Monetdb-developers] Restart mclient
by Romulo Antonio Goncalves 07 Sep '10

07 Sep '10

Hello, I updated my default branch and I do the following: mclient -lmal >\q mclient -lmal >\q After the second exit I get a crash in the mserver: >[New Thread 0x7fffdf7ac710 (LWP 19441)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffdf7ac710 (LWP 19441)] 0x00007ffff7cd7198 in MALexitClient (c=0x605b38) at /ufs/goncalve/MonetDB/current/MonetDB5/src/mal/mal_session.mx:522 522 if (c->glb && c->curprg->def->errors == 0) (gdb) p c->curprg->def $1 = (struct MALBLK *) 0xdbdbdbdbdbdbdbdb Similar problem if you do mclient -lsql >\q mclient -lsql The mserver crashes due to the same problem: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffdf7ac710 (LWP 19561)] 0x00007fffdfe22a60 in SQLparser (c=0x605b38) at /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_scenario.mx:1155 1155 oldvtop = c->curprg->def->vtop; (gdb) p c->curprg $5 = (Symbol) 0x7fffd8112198 (gdb) p *c->curprg $6 = {peer = 0xdbdbdbdbdbdbdbdb, skip = 0xdbdbdbdbdbdbdbdb, name = 0xdbdbdbdbdbdbdbdb <Address 0xdbdbdbdbdbdbdbdb out of bounds>, kind = -606348325, def = 0xdbdbdbdbdbdbdbdb} (gdb) Were there any changes in the client structure or in the initialization of the of the client threads? Regards, Romulo PS: It does not happen in the Oct2010 branch.

3 2

[Monetdb-developers] How to enable functions via sql?
by Alexandr Kulik 06 Sep '10

06 Sep '10

Hi, I need to develop some extra statistical functions for MonetDB. I spent about a week to understand how to use function via sql, but still didn't have any succes results. May be someone could point me to appropriate documentation or source revision, where i could look on it? -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

2 1

[Monetdb-developers] New Oct2010 stable branch created
by Sjoerd Mullender 03 Sep '10

03 Sep '10

I have created new stable branches for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. The name for the branch is Oct2010. So in each of your stable checkout directories you can do hg pull hg update -rOct2010 to update to this new branch. -- Sjoerd Mullender

1 0

[Monetdb-developers] About MonetDB/X100
by xixi 01 Sep '10

01 Sep '10

Dear all, Recently I read some papers about X100,and see that the performance of X100 is very good ,and better than MonetDB/sql, and have a better design,so I want to ask when will we can see the code of the X100 ,and use the X100? Best regards!

2 1

[Monetdb-developers] Announcement: New Jun2010-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 30 Aug '10

30 Aug '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010-SP2 bug fix release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] Testing update performance of MonetDB
by Lukas Kircher 25 Aug '10

25 Aug '10

Hi, I'm currently writing my bachelor's thesis at the university of Konstanz about extending BaseX [1] with XQuery Update. I wanted to compare evaluation times between MonetDB and BaseX, as they have comparable table layouts, but am facing several problems. So far I read your manual that suggests to use functions to support caching, as well as limiting inserts to a single position in a document. Some papers provided insight too [2,3], still there are questions (hopefully I didn't miss anything on your mailing lists and I hit the right one): 1. General storage: In which way is free space reserved, if an updatable document via fn:add-doc(u, x, y, freeSpace) is added? Let's assume freeSpace = 10%. Does this mean that free space (equal to 10% of our document size) is allocated between all used pages on disk? Spoken clearly, one tenth of free space on every page? But this would be in conflict with the suggestion to limit inserts to a single position in a document... 2. Timing: What happens during the 'Print'...ing part when I use the '-t' flag for timings? Query time is amazingly fast, yet I experience unusually long printing times - even for a small test document. Increasing the free space variable (see question 1) results in even longer printing times, which is on one side perfectly understandable as more pages on disk are used (right?)... 3. Yet, on the other side empty space on disk should not be accessed? I expect there to be a balance between query performance and free space? Is there a rule of thumb I can apply to get the most out of MonetDB (regarding specific queries of course)? 4. Are there any general advises for testing update performance of MonetDB? I could gain some knowledge from a previous post on the developers list - yet I feel it's not enough to explain the results of my (yet immature) tests and to do your system justice ... Thanks for your help! Looking forward to your answer... Best regards, Lukas [1] www.basex.org [2] Updating the Pre/Post Plane in MonetDB/XQuery [3] MonetDB/XQuery: A Fast XQuery Processor Powered by a Relational Engine

1 0

[Monetdb-developers] About MAL instrction and C-function
by 嘻嘻 23 Aug '10

23 Aug '10

Dear all, Recently, I am learning the MonetDB,I know that a MAL instrction is corresponding to a C-function. such as a MAL instrction algebra.select() is corresponding to ALGselectInclusive(); and the corresponding file is algebra.mal, for example, command select(b:bat[:any_1,:any_2], low:any_2, high:any_2, li:bit, hi:bit) :bat[:any_1,:any_2] address ALGselectInclusive comment "Select all BUNs that have tail values: {v| low <{=} v <{=} high}. Boundary inclusion is indicated separately. NIL boundary values have a special meaning. + low == nil means: no lower bound + high == nil means: no upper bound."; My question is ,in MonetDB souces code, how to corresponding the MAL to the C-function, how to implement it ? and how to transfer the parameter from MAL to C-function? Best regards!

1 0

[Monetdb-developers] run mapi examples
by kun ren 19 Aug '10

19 Aug '10

dear all recently,I want to run mapi examples from /Monetdb-client/clients/src/examples/C/, it contains sample0.c ,sample1.c and so on. I read the Application Programming Interfaces of the Monetdb manual, and I use: gcc sample0.c 'monetdb-clients-config --cflags --lib' -lMapi -o sample0 to complie it,but it has errors, so I want to ask how I will complie the files? Best regards!

2 4

[Monetdb-developers] about MAL instrction
by kun ren 18 Aug '10

18 Aug '10

Dear all, I use MAL interface to manipulate the BAT data, I use sql.bind(,,0) to get a BAT, and use io.print() to print the BAT content, but when I use bat.insert to insert one bun to the bat,but after the bat.insert(), I run io.print() to print the BAT,but the bat content is not changed，I use sql.bind(,,,,1) to get the insert BAT of the orinal bat， but it is null. I think it it about commit() ?? how I commit it ? do I think right? Best regards!

1 0

[Monetdb-developers] about Using Mal
by kun ren 18 Aug '10

18 Aug '10

Dear all, Recently,I am using MAL,using mclient -l mal command, If I use sql, I write a SQL: select * from t1; while t1 has two column c1 and c2,type is int. I want to ask what the corresponding MAL of this sql, if I us mclient -l mal ,what mal will I will use to implement the select * from t1 Best regards!

3 8

[Monetdb-developers] about implement some application interface
by kun ren 16 Aug '10

16 Aug '10

Dear all, My test application want to implement two application interface over MonetDB , one is loading *.csv file to MonetDB, to implement this interface ,I use MonetDB's JDBC and "copy from" command to implement it. and another interface is select from one bat using select conditions, such as this C function: bat *selectfrombat(char * batname,char * select condition), how do I implement it?do MonetDB has something like "copy from " command,and so I will use jdbc and the command to implement it? or other method to implement it? I know one MAL instruction is algebra.uselect, and corresponding C function, do I just use the C function or alter the C fuction? Best regards,

1 0

[Monetdb-developers] Merge Tables MAL optimizer
by kun ren 13 Aug '10

13 Aug '10

dear all， Recently when I am reading MAL optimizer manual of MonetDB.and don't know the Merge Table optimizer,A merge association table (MAT). (1)How it perform optimize the MAL? (2) for this example, before optimize, m0:= bat.new(:oid,:int); m1:= bat.new(:oid,:int); m2:= bat.new(:oid,:int); b := mat.new(m0,m1,m2); s := algebra.select(b,1,3); i := aggr.count(s); io.print(s); io.print(i); c0 := bat.new(:int,:int); c1 := bat.new(:int,:int); c := mat.new(c0,c1); j := algebra.join(b,c); io.print(j); The selection and aggregate operations can simply be rewritten using a MAT: _33 := algebra.select(m0,1,3); _34 := algebra.select(m1,1,3); _35 := algebra.select(m2,1,3); s := mat.new(_33,_34,_35); i := 0:int; _36 := aggr.count(_33); i := calc.+(i,_36); Chapter 5: The MAL Optimizer 90 _37 := aggr.count(_34); i := calc.+(i,_37); _38 := aggr.count(_35); i := calc.+(i,_38); io.print(i); It seems that mat is very same with the bat.kunion, but what different between them, and how it optimizer the MAL? Best regards!

1 0

[Monetdb-developers] about MAL optimize
by kun ren 11 Aug '10

11 Aug '10

Dear everyone, Recently,I am intersting in MAL optimize，and have read the manual of the MAL optimize, but it is so long, Possible if we want to use mal optimize ,we must degignation the optimize consideration, such as alias removal, constant expression,join path optimizer;And then I dedug the MonetDB,but default it don't use any optimize.so what do you think about the mal optimize? Best regards.

3 9

[Monetdb-developers] About monetdb's thread
by kun ren 30 Jul '10

30 Jul '10

Dear all, hello,all, I download the windows installer of monetdb,and install,and then start MonetDB server, my computer has 4 cpus.and with 32bit windows xp ,but the MonetDB server only start one thread.when I run MonetDB in Linux with 2 cpus, it start 2 threads. so why? why I am in Windows xp with 4 cpus, it only create 1 thread ?

3 3

[Monetdb-developers] need help for extracting BAT
by Gaetano Fabiano 29 Jul '10

29 Jul '10

Hi all, i'm a new user of MonetDB. I worked on MonetDB/SQL and I can use well it. Now I need to use MonetDB directly with MAL. But i'm not so able to do that. I run the client with this command on linux: mclient -lmal -d database So I can write MAL istruction. But I don't need to create a procedure in MAL that doesn't access to my data. What I need to do is that: I need to give the BAT data of my DataBase and put all data on my application: example C++ ora Java array, after that I need to manipulate this BAT with my application and update the original BAT. How can I do that? summarizing i need: - Access to the BAT of the table of my Database. - Print it on monitor - Put it in array, file, stack, or any DataStruct with any programming language. - Set my manipulated BAT again in MonetDB. All of this is possible? Someone can help me? I'm sorry if I don't know the netiquette of your mailing list but I need to do that and I don't know where I could be helped. Thanks. Gaetano.

2 1

Re: [Monetdb-developers] MonetDB: Jun2010 - It seems the err output for the test was not ...
by Stefan Manegold 22 Jul '10

22 Jul '10

On Thu, Jul 22, 2010 at 11:53:22AM +0200, Romulo Goncalves wrote: > Changeset: 7a88e64c34b1 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=7a88e64c34b1 > Modified Files: > sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.err > sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.out > Branch: Jun2010 > Log Message: > > It seems the err output for the test was not complete. > In the approval of the test two sql statements were missing, therefore, the err output was created without the errors related to that two statements. Thanks for taking care of this. My reason for not approving the error messages in the first place was that I was not sure about the intention of the test, and hence, whether and which error messages are to be expected. I now understand from your approval that the error messages indeed match the intention of the test. I wonder, though, whether the error message is indeed what it should be. Saying "failed: Success" (see below) seems at least "odd" to me ... Stefan > > diffs (32 lines): > > diff -r 6908fb1d95e4 -r 7a88e64c34b1 sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.err > --- a/sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.err Thu Jul 22 11:02:07 2010 +0200 > +++ b/sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.err Thu Jul 22 11:53:14 2010 +0200 > @@ -71,9 +71,15 @@ > # 13:12:55 > mclient -lsql -ftest -i -e --host=rig --port=39884 > # 13:12:55 > > > -MAPI = monetdb@rig:39884 > +MAPI = monetdb@alviss:36310 > QUERY = connect to default; > ERROR = !CONNECT TO: DEFAULT is not supported! > +MAPI = monetdb@alviss:36310 > +QUERY = connect to 'whatever' port 50001 database 'nonexisting' USER 'monetdb' PASSWORD 'monetdb' LANGUAGE 'mal'; > +ERROR = !IOException:mapi.connect:Could not connect: gethostbyname failed: Success ^^^^^^ ^^^^^^^ !!!!!! ??????? > +MAPI = monetdb@alviss:36310 > +QUERY = disconnect 'whatever'; > +ERROR = !DISCONNECT CATALOG: no such db_alias 'whatever' > > # 13:12:55 > > # 13:12:55 > Done. > diff -r 6908fb1d95e4 -r 7a88e64c34b1 sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.out > --- a/sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.out Thu Jul 22 11:02:07 2010 +0200 > +++ b/sql/src/test/BugTracker-2010/Tests/connectto.Bug-2548.stable.out Thu Jul 22 11:53:14 2010 +0200 > @@ -23,8 +23,6 @@ > # 13:12:55 > mclient -lsql -ftest -i -e --host=rig --port=39884 > # 13:12:55 > > > -! to be provided / approved ! > - > # 13:12:55 > > # 13:12:55 > Done. > # 13:12:55 > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] about debug monetdb on windows
by kun ren 20 Jul '10

20 Jul '10

Dear everyone, Some days ago,I debug monetdb on linux ,use eclipse + gdb, but now I want to debug and compile the monetdb on windows, please tell me how would i will do to compile and debug the monetdb. Best regards!

2 1

[Monetdb-developers] about copy command of Monetdb
by kun ren 20 Jul '10

20 Jul '10

Dear all, Recently,I use TPC-H test the Monetdb, On windows ,so I use "copy ,, records,,,into,,from" to load data to Monetdb,when the size of the file is about 700M,It is OK.But when the size is 1.4G,the Monetdb server is auto stop when loading. But when I am in Linux, it is no problem. So I want to ask why it is wrong on windows when the file is so large? Best regards,

1 0

[Monetdb-developers] Announcement: New Jun2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 17 Jul '10

17 Jul '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010-SP1 bug fix release of the MonetDB suite of programs. Lots of problems have been fixed, the most important one being the fix in the handling of database upgrades of databases created with the Feb2010 release to the current version. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location has changed to <http://dev.monetdb.org/downloads/>. Please fix any bookmarks you may have. -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] RDF data management in MonetDB/SQL
by wangx 17 Jul '10

17 Jul '10

Dear Lefteris, Thank you for your reply. Your explanation will help me much. In addition, would you give me some guidelines on how to make the 6 different indices of the triple table be understood by the SQL/SPARQL query optimizer and used accordingly. If I want to implement this feature by myself, in which files the code should be added or modified. BTW, in the future, will you implement the MonetDB/SPARQL as a new supported front-end language? Will you consider to translate SPARQL to MAL directly, instead of through SQL to MAL (but this way you have to develop a SPARQL specific optimizer, instead of making use of the SQL optimizer)? Thank you so much. Best regards, Xin Wang >From: Lefteris <lsidir(a)gmail.com> >Reply-To: >To: wangx <wangx(a)tju.edu.cn> >Subject: Re: [Monetdb-developers] RDF data management in MonetDB/SQL >Date:Fri, 16 Jul 2010 10:29:53 +0200 > >Hi, > >your understanding is in general correct. The availability of the six >different indices of the triple table should be understood by the >SQL/SPARQL query optimizer and used accordingly. However, the >MonetDB/RDF module has not been announced yet, and that is because it >has not finished yet. You are using a piece of code which is >experimental and unfinished. As such, the only way to test Monet and >its capabilities on RDF data is to manually write the SQL query in >such a way that you use the correct order of the triple table on the >correct join. > >In the future of course, this will be done by the optimizer and the >user will only have to write simple SPARQL queries referring to the >name of the graph, instead of the underlying storage schema, but until >then I am afraid you will have to do it by hand. The good news is that >if you are using RDF data and testing queries that have been published >previously on papers as benchmarks, most likely someone else will have >done already the translation to a correct SQL query (since most >experiments on newly build engines that do not support SPARQL use this >method). > >Hope this helps you a bit, > >lefteris > >2010/7/15 wangx : >> Hi MonetDB developers, >> I have a question about RDF data management in MonetDB/SQL. The comment of >> sql.rdfshred says "shredding an RDF data file from location results in 7 new >> tables (6 permutations of SPO and a mapping) ... We can then query with SQL >> queries the RDF triple store by quering tables gid_spo, gid_pso etc., ...". >> In my option, if the spo table is considered the triples table, the other 5 >> tables (sop, pso, pos, osp, ops) (except the mapping table) can be viewed as >> indexes of the triples table spo. >> When I writeSQLto query the shredded RDF data in the triples table, I >> have two ways.The first way is toonly use spo table tomake self-joins. >> The second way is touse all 6 tables to make joins. I noticed that >> "MonetDB/SQL Reference Manual" says that "The heart is the MonetDB server, >> which comes with the following innovative features. ... Index selection, >> creation and maintenance is automatic". IfI use6 tables (as indexes) >> explicitly to make joins, it seems that I write the query plan by myself. >> However, I think this work should be done by the SQL optimizer using >> statistics from the system catalog. I wondered if these tables have >> alreadybeen specified as indexes in the internal code, or if there is a way >> to specify it so that the optimizer can use them as indexes to generate >> query plans. I am not sure if my understanding is correct. I will appreciate >> any help from developers. Thank you in advance. >> >> Best regards, >> Xin Wang >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Sprint >> What will you do first with EVO, the first 4G phone? >> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first >> _______________________________________________ >> Monetdb-developers mailing list >> Monetdb-developers(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >> >> >

2 1

Re: [Monetdb-developers] MonetDB: default - implement timing per server response for resu...
by Stefan Manegold 16 Jul '10

16 Jul '10

On Thu, Jul 15, 2010 at 03:25:39PM +0200, Fabian Groffen wrote: > Changeset: dfcab86b0cb0 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=dfcab86b0cb0 > Modified Files: > clients/ChangeLog > clients/src/mapiclient/mclient.mx > Branch: default > Log Message: > > implement timing per server response for results and updates for SQL formatter > [...] > diff -r df828b8fc84c -r dfcab86b0cb0 clients/src/mapiclient/mclient.mx > --- a/clients/src/mapiclient/mclient.mx Thu Jul 15 15:09:22 2010 +0200 > +++ b/clients/src/mapiclient/mclient.mx Thu Jul 15 15:11:07 2010 +0200 > @@ -342,6 +342,26 @@ > } > } > > +static char htimbuf[32]; > +static char * > +timerHuman() > +{ > + long t = t1 - t0; > + > + if (t / 1000 < 950) { > + snprintf(htimbuf, 32, "%ld.%03ldms", t / 1000, t % 1000); > + return(htimbuf); > + } > + t /= 1000; > + if (t / 1000 < 60) { > + snprintf(htimbuf, 32, "%ld.%02lds", t / 1000, t % 1000); > + return(htimbuf); > + } > + t /= 1000; > + snprintf(htimbuf, 32, "%ldm %lds", t / 60, t % 60); > + return(htimbuf); > +} > + > /* The Mapi library eats away the comment lines, which we need to > * detect end of debugging. We overload the routine to our liking. */ > [...] While I like the new timing feature as well as the convenient formatting for human readability, I am wondering about two potential changes / extensions: (a) While the time formatting is convenient for human readbility, it might be less convenient for automatic (post-)processing, e.g., creating performance figures. For that, plain unformatted microseconds (in addition to the human-readable format) might be more convenient. (b) The "old" (still existing) "Timer" information is sent to stderr, thus, one can seem the Timer output without the need to grep through (potentially large) output, e.g., by simply redirecting the output to a file or even /dev/null --- obviously, aligning stderr & stdout in case of large multi-query scripts is then left to the user. The new timing output goes (together with the queries status summary like how many result tuples were produced) to stdout (only). (c) As the announcement (changelog) says, the new timing output is only produced with the "sql" formatter (i.e., not with, e.g., "raw" or "test", and hence, also not by default with `-s <sql-statement>` (unless one explicitly requests `-f sql`). For (a) & (b) would it be an option to consider to also send the plain unformatted times in microseconds to stderr to (better) support automatic post-processing of the timings? To cover also (c), we could consider doing that with all formatters, not only with "sql". ... just some ideas to think about and discuss ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

3 3

[Monetdb-developers] RDF data management in MonetDB/SQL
by wangx 16 Jul '10

16 Jul '10

Hi MonetDB developers, I have a question about RDF data management in MonetDB/SQL. The comment of sql.rdfshred says "shredding an RDF data file from location results in 7 new tables (6 permutations of SPO and a mapping) ... We can then query with SQL queries the RDF triple store by quering tables gid_spo, gid_pso etc., ...". In my option, if the spo table is considered the triples table, the other 5 tables (sop, pso, pos, osp, ops) (except the mapping table) can be viewed as indexes of the triples table spo. When I write SQL to query the shredded RDF data in the triples table, I have two ways. The first way is to only use spo table to make self-joins. The second way is to use all 6 tables to make joins. I noticed that "MonetDB/SQL Reference Manual" says that "The heart is the MonetDB server, which comes with the following innovative features. ... Index selection, creation and maintenance is automatic". If I use 6 tables (as indexes) explicitly to make joins, it seems that I write the query plan by myself. However, I think this work should be done by the SQL optimizer using statistics from the system catalog. I wondered if these tables have already been specified as indexes in the internal code, or if there is a way to specify it so that the optimizer can use them as indexes to generate query plans. I am not sure if my understanding is correct. I will appreciate any help from developers. Thank you in advance. Best regards, Xin Wang

2 1

[Monetdb-developers] TPC-H benchmark about MonetDB
by kun ren 16 Jul '10

16 Jul '10

Dear everyone, Recently,i am testing the performance of MonetDB. But I want to know the time of execution one sql. I type \t on CLI. and then it returns a execution time. But I found that the time include the print resultset time.I just only want to know the time of execution ,not include the print time. So please tell me what do i do? -----------------Best regards!

3 7

[Monetdb-developers] TPC-H Query-14
by Romulo Goncalves 15 Jul '10

15 Jul '10

Hello, I am running TPC-H in the default version, updated 1 minute ago. Compilation in Debug mode and I am running in my Desktop at CWI. I loaded TPC-H SF-10 without constraints. [goncalve@alviss queries]$ mclient -lsql < 14.sql MAPI = monetdb@localhost:55000 QUERY = select ERROR = !SQLException:convert:too many digits (20 > 19) #select * from optimizer_stats() stats; % .stats, .stats # table_name % rewrite, count # name % clob, int # type % 7, 1 # length [ "joinidx", 0 ] [goncalve@alviss queries]$ Why do I get an Exception when I got a result? Regards, Romulo

3 2

[Monetdb-developers] bug #2341 [was: MonetDB scalability]
by Mark Bucciarelli 15 Jul '10

15 Jul '10

[Moving thread from -users to -dev.] On Thu, Jul 8, 2010 at 6:28 PM, Niels Nes <Niels.Nes(a)cwi.nl> wrote: > On Thu, Jul 08, 2010 at 03:05:35PM -0400, Mark Bucciarelli wrote: >> >> Dropping a column corrupts DB (#2341). >> Same as above. > > Both were fixed some time ago and tests were added by sjoerd. > These run fine, ie I closed them. > By "some time ago" you must mean post-Feb2010 release. I just corrupted my database by deleting a column and a constraint. Could you link the changeset to the ticket so I can apply the patch manually? If you applied fix to Feb2010 branch, please link that changeset, as that's what I am running. Thanks, m

1 0

Re: [Monetdb-developers] MonetDB: Jun2010 - disable parallelism (mitosis & dataflow) to a...
by Stefan Manegold 13 Jul '10

13 Jul '10

This changeset accientally contains changes to file sql/src/server/sql_parser.mx that should have gone into a seperate checking / changeset: " Refined my earlier fix "fixed (restored) order_by over union_all" (http://dev.monetdb.org/hg/MonetDB/rev/35ebc65eb74c) we now also (re-)allow ORDER BY over EXCEPT & UNION . Note. though, that (as also in Feb2010 and most probably earlier releases, MonetDB incorrectly ignores ORDER BY over UNION, EXCEPT & INTERSECT. See also Bug 2606 (http://bugs.monetdb.org/2606) " Stefan On Tue, Jul 13, 2010 at 10:16:39AM +0200, Stefan Manegold wrote: > Changeset: e399a0ffae31 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e399a0ffae31 > Modified Files: > sql/src/server/sql_parser.mx > sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.sql > sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Debian > sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Windows > sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.sql > sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out > sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out.Debian > sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out.Windows > sql/src/test/BugTracker-2010/Tests/group-by_ordered_column.Bug-2564.sql > sql/src/test/BugTracker-2010/Tests/group-by_ordered_column.Bug-2564.stable.out > sql/src/test/BugTracker-2010/Tests/group-by_ordered_column.Bug-2564.stable.out.32bit > sql/src/test/BugTracker/Tests/explain.SF-1739353-data.sql > sql/src/test/BugTracker/Tests/explain.SF-1739353.stable.out > sql/src/test/Tests/trace.sql > sql/src/test/Tests/trace.stable.out > Branch: Jun2010 > Log Message: > > disable parallelism (mitosis & dataflow) to avoid ambiguous EXPLAIN results > > > diffs (truncated from 630 to 300 lines): > > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/server/sql_parser.mx > --- a/sql/src/server/sql_parser.mx Tue Jul 13 01:09:49 2010 +0200 > +++ b/sql/src/server/sql_parser.mx Tue Jul 13 10:16:33 2010 +0200 > @@ -2922,7 +2922,11 @@ > { > $$ = $1; > if ($2 || $3 || $4) { > - if ($1 != NULL && ($1->token == SQL_SELECT || $1->token == SQL_UNION)) { > + if ($1 != NULL && > + ($1->token == SQL_SELECT || > + $1->token == SQL_UNION || > + $1->token == SQL_EXCEPT || > + $1->token == SQL_INTERSECT)) { > SelectNode *s = (SelectNode*)$1; > > s -> orderby = $2; > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.sql > --- a/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.sql Tue Jul 13 01:09:49 2010 +0200 > +++ b/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.sql Tue Jul 13 10:16:33 2010 +0200 > @@ -1,3 +1,5 @@ > +-- disable parallelism (mitosis & dataflow) to avoid ambiguous results > +set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,history,multiplex,garbageCollector'; > create table SF_2596114 (x double); > explain select power(x,2) from SF_2596114; > explain select x*x from SF_2596114; > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > --- a/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out Tue Jul 13 01:09:49 2010 +0200 > +++ b/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out Tue Jul 13 10:16:33 2010 +0200 > @@ -31,7 +31,6 @@ > % 0 # length > function user.s0_1{autoCommit=true}():void; > _2 := sql.mvc(); > -barrier _36 := language.dataflow(); > _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > _10 := algebra.kdifference(_3,_8); > @@ -42,7 +41,6 @@ > _16 := bat.reverse(_15); > _17 := algebra.kdifference(_14,_16); > _18:bat[:oid,:dbl] := batmmath.pow(_17,2); > -exit _36; > _19 := sql.resultSet(1,1,_18); > sql.rsColumn(_19,"sys.","power_x","double",53,0,_18); > _25 := io.stdout(); > @@ -54,7 +52,6 @@ > % 0 # length > function user.s1_1{autoCommit=true}():void; > _2 := sql.mvc(); > -barrier _35 := language.dataflow(); > _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > _10 := algebra.kdifference(_3,_8); > @@ -65,7 +62,6 @@ > _16 := bat.reverse(_15); > _17 := algebra.kdifference(_14,_16); > _18:bat[:oid,:dbl] := batcalc.*(_17,_17); > -exit _35; > _19 := sql.resultSet(1,1,_18); > sql.rsColumn(_19,"sys.","sql_mul_x","double",53,0,_18); > _25 := io.stdout(); > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Debian > --- a/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Debian Tue Jul 13 01:09:49 2010 +0200 > +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 > @@ -1,74 +0,0 @@ > -stdout of test 'POWER_vs_prod.SF-2596114` in directory 'src/test/BugTracker-2009` itself: > - > - > -# 23:02:29 > > -# 23:02:29 > mserver5 "--config=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/etc/monetdb5.conf" --debug=10 --set gdk_nr_threads=0 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5/lib:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB5/sql_logs" --set mapi_open=true --set xrp > c_open=true --set mapi_port=33742 --set xrpc_port=44239 --set monet_prompt= --trace "--dbname=mTests_src_test_BugTracker-2009" --set mal_listing=0 "--dbinit= include sql;" ; echo ; echo Over.. > -# 23:02:29 > > - > -# MonetDB server v5.10.3, based on kernel v1.28.3 > -# Serving database 'mTests_src_test_BugTracker-2009', using 4 threads > -# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > -# Copyright (c) 1993-July 2008 CWI. > -# Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved > -# Visit http://monetdb.cwi.nl/ for further information > -# Listening for connection requests on mapi:monetdb://rig.ins.cwi.nl:33742/ > -# MonetDB/SQL module v2.28.1 loaded > - > -Ready. > -#function user.main():void; > -# clients.quit(); > -#end main; > - > -Over.. > - > -# 15:46:08 > > -# 15:46:08 > mclient -lsql -i -umonetdb -Pmonetdb --host=rig --port=35464 > -# 15:46:08 > > - > -% .explain # table_name > -% mal # name > -% clob # type > -% 0 # length > -function user.s0_1{autoCommit=true}():void; > - _2 := sql.mvc(); > - _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > - _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > - _10 := algebra.kdifference(_3,_8); > - _11 := algebra.kunion(_10,_8); > - _12:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",1); > - _14 := algebra.kunion(_11,_12); > - _15:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","sf_2596114",1); > - _16 := bat.reverse(_15); > - _17 := algebra.kdifference(_14,_16); > - _18:bat[:oid,:dbl] := batmmath.pow(_17,2); > - _19 := sql.resultSet(1,1,_18); > - sql.rsColumn(_19,"sys.","power_x","double",53,0,_18); > - _25 := io.stdout(); > - sql.exportResult(_25,_19); > -end s0_1; > -% .explain # table_name > -% mal # name > -% clob # type > -% 0 # length > -function user.s1_1{autoCommit=true}():void; > - _2 := sql.mvc(); > - _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > - _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > - _10 := algebra.kdifference(_3,_8); > - _11 := algebra.kunion(_10,_8); > - _12:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",1); > - _14 := algebra.kunion(_11,_12); > - _15:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","sf_2596114",1); > - _16 := bat.reverse(_15); > - _17 := algebra.kdifference(_14,_16); > - _18:bat[:oid,:dbl] := batcalc.*(_17,_17); > - _19 := sql.resultSet(1,1,_18); > - sql.rsColumn(_19,"sys.","sql_mul_x","double",53,0,_18); > - _25 := io.stdout(); > - sql.exportResult(_25,_19); > -end s1_1; > - > -# 23:02:30 > > -# 23:02:30 > Done. > -# 23:02:30 > > - > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Windows > --- a/sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out.Windows Tue Jul 13 01:09:49 2010 +0200 > +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 > @@ -1,74 +0,0 @@ > -stdout of test 'POWER_vs_prod.SF-2596114` in directory 'src/test/BugTracker-2009` itself: > - > - > -# 23:02:29 > > -# 23:02:29 > mserver5 "--config=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/etc/monetdb5.conf" --debug=10 --set gdk_nr_threads=0 --set "monet_mod_path=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5/lib:/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/lib64/MonetDB5/bin" --set "gdk_dbfarm=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB5/dbfarm" --set "sql_logdir=/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB5/sql_logs" --set mapi_open=true --set xrp > c_open=true --set mapi_port=33742 --set xrpc_port=44239 --set monet_prompt= --trace "--dbname=mTests_src_test_BugTracker-2009" --set mal_listing=0 "--dbinit= include sql;" ; echo ; echo Over.. > -# 23:02:29 > > - > -# MonetDB server v5.10.3, based on kernel v1.28.3 > -# Serving database 'mTests_src_test_BugTracker-2009', using 4 threads > -# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > -# Copyright (c) 1993-July 2008 CWI. > -# Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved > -# Visit http://monetdb.cwi.nl/ for further information > -# Listening for connection requests on mapi:monetdb://rig.ins.cwi.nl:33742/ > -# MonetDB/SQL module v2.28.1 loaded > - > -Ready. > -#function user.main():void; > -# clients.quit(); > -#end main; > - > -Over.. > - > -# 15:46:08 > > -# 15:46:08 > mclient -lsql -i -umonetdb -Pmonetdb --host=rig --port=35464 > -# 15:46:08 > > - > -% .explain # table_name > -% mal # name > -% clob # type > -% 0 # length > -function user.s0_1{autoCommit=true}():void; > - _2 := sql.mvc(); > - _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > - _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > - _10 := algebra.kdifference(_3,_8); > - _11 := algebra.kunion(_10,_8); > - _12:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",1); > - _14 := algebra.kunion(_11,_12); > - _15:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","sf_2596114",1); > - _16 := bat.reverse(_15); > - _17 := algebra.kdifference(_14,_16); > - _18:bat[:oid,:dbl] := batmmath.pow(_17,2); > - _19 := sql.resultSet(1,1,_18); > - sql.rsColumn(_19,"sys.","power_x","double",53,0,_18); > - _25 := io.stdout(); > - sql.exportResult(_25,_19); > -end s0_1; > -% .explain # table_name > -% mal # name > -% clob # type > -% 0 # length > -function user.s1_1{autoCommit=true}():void; > - _2 := sql.mvc(); > - _3:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",0); > - _8:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",2); > - _10 := algebra.kdifference(_3,_8); > - _11 := algebra.kunion(_10,_8); > - _12:bat[:oid,:dbl] := sql.bind(_2,"sys","sf_2596114","x",1); > - _14 := algebra.kunion(_11,_12); > - _15:bat[:oid,:oid] := sql.bind_dbat(_2,"sys","sf_2596114",1); > - _16 := bat.reverse(_15); > - _17 := algebra.kdifference(_14,_16); > - _18:bat[:oid,:dbl] := batcalc.*(_17,_17); > - _19 := sql.resultSet(1,1,_18); > - sql.rsColumn(_19,"sys.","sql_mul_x","double",53,0,_18); > - _25 := io.stdout(); > - sql.exportResult(_25,_19); > -end s1_1; > - > -# 23:02:30 > > -# 23:02:30 > Done. > -# 23:02:30 > > - > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.sql > --- a/sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.sql Tue Jul 13 01:09:49 2010 +0200 > +++ b/sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.sql Tue Jul 13 10:16:33 2010 +0200 > @@ -1,3 +1,5 @@ > +-- disable parallelism (mitosis & dataflow) to avoid ambiguous results > +set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,history,multiplex,garbageCollector'; > \f csv > create table blabla(id integer); > select '#~BeginVariableOutput~#'; > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out > --- a/sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out Tue Jul 13 01:09:49 2010 +0200 > +++ b/sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out Tue Jul 13 10:16:33 2010 +0200 > @@ -28,8 +28,7 @@ > #~BeginVariableOutput~# > function user.s1_1{autoCommit=true}():void; > _2 := sql.mvc(); > -" sql.catalog(29\,\"sys\"\,139834807776504\,0);" > -barrier _37 := language.dataflow(); > +" sql.catalog(29\,\"sys\"\,140688029790296\,0);" > " _7:bat[:oid\,:int] := sql.bind(_2\,\"sys\"\,\"blabla\"\,\"id\"\,0);" > _10 := algebra.selectNotNil(_7); > " _11:bat[:oid\,:oid] := sql.bind_dbat(_2\,\"sys\"\,\"blabla\"\,1);" > @@ -43,13 +42,11 @@ > _21 := calc.isnil(_20); > " _23 := calc.ifthenelse(_21\,0:wrd\,_20);" > " _24 := calc.!=(_19\,_23);" > -exit _37; > " sql.assert(_24\,\"UPDATE: UNIQUE constraint 'blabla.dada' violated\");" > end s1_1; > function user.s2_1{autoCommit=true}():void; > _2 := sql.mvc(); > -" sql.catalog(29\,\"sys\"\,139834808040824\,0);" > -barrier _37 := language.dataflow(); > +" sql.catalog(29\,\"sys\"\,140688030052568\,0);" > " _7:bat[:oid\,:int] := sql.bind(_2\,\"sys\"\,\"blabla\"\,\"id\"\,0);" > _10 := algebra.selectNotNil(_7); > " _11:bat[:oid\,:oid] := sql.bind_dbat(_2\,\"sys\"\,\"blabla\"\,1);" > @@ -63,15 +60,13 @@ > _21 := calc.isnil(_20); > " _23 := calc.ifthenelse(_21\,0:wrd\,_20);" > " _24 := calc.!=(_19\,_23);" > -exit _37; > " sql.assert(_24\,\"UPDATE: UNIQUE constraint 'blabla.dada' violated\");" > end s2_1; > #~EndVariableOutput~# > #~BeginVariableOutput~# > function user.s1_1{autoCommit=true}():void; > _2 := sql.mvc(); > -" sql.catalog(29\,\"sys\"\,139834807056504\,0);" > -barrier _37 := language.dataflow(); > +" sql.catalog(29\,\"sys\"\,140688029790296\,0);" > " _7:bat[:oid\,:int] := sql.bind(_2\,\"sys\"\,\"blabla\"\,\"id\"\,0);" > _10 := algebra.selectNotNil(_7); > " _11:bat[:oid\,:oid] := sql.bind_dbat(_2\,\"sys\"\,\"blabla\"\,1);" > @@ -85,7 +80,6 @@ > _21 := calc.isnil(_20); > " _23 := calc.ifthenelse(_21\,0:wrd\,_20);" > " _24 := calc.!=(_19\,_23);" > -exit _37; > " sql.assert(_24\,\"UPDATE: UNIQUE constraint 'blabla.dada' violated\");" > end s1_1; > #~EndVariableOutput~# > diff -r 75116ec8f9a7 -r e399a0ffae31 sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out.Debian > --- a/sql/src/test/BugTracker-2009/Tests/explain_gives_crash.SF-2741829.stable.out.Debian Tue Jul 13 01:09:49 2010 +0200 > +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 > @@ -1,90 +0,0 @@ > -stdout of test 'explain_gives_crash.SF-2741829` in directory 'src/test/BugTracker-2009` itself: > - > - > -# 22:45:54 > > -# 22:45:54 > mserver5 "--config=/ufs/niels/scratch/rc/Linux-x86_64/etc/monetdb5.conf" --debug=10 --set gdk_nr_threads=0 --set "monet_mod_path=/ufs/niels/scratch/rc/Linux-x86_64/lib/MonetDB5:/ufs/niels/scratch/rc/Linux-x86_64/lib/MonetDB5/lib:/ufs/niels/scratch/rc/Linux-x86_64/lib/MonetDB5/bin" --set "gdk_dbfarm=/ufs/niels/scratch/rc/Linux-x86_64/var/MonetDB5/dbfarm" --set mapi_open=true --set xrpc_open=true --set mapi_port=35519 --set xrpc_port=48894 --set monet_prompt= --trace "--dbname=mTests_src_test_BugTracker-2009" --set mal_listing=0 "--dbinit= include sql;" ; echo ; echo Over.. > -# 22:45:54 > > - > -# MonetDB server v5.12.0, based on kernel v1.30.0 > -# Serving database 'mTests_src_test_BugTracker-2009', using 4 threads > -# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked > -# Copyright (c) 1993-July 2008 CWI. > -# Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved > -# Visit http://monetdb.cwi.nl/ for further information > -# Listening for connection requests on mapi:monetdb://alf.ins.cwi.nl:35519/ > -# MonetDB/SQL module v2.30.0 loaded > - > -Ready. > -#function user.main():void; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

Re: [Monetdb-developers] about import *.csv
by Stefan de Konink 12 Jul '10

12 Jul '10

On Mon, 12 Jul 2010, kun ren wrote: > I am new to Monetdb, I want to ask how to excute the statement mclient -lsql >> if i want to import many *.csv,such as a.csv,b.csv,,,, but i only have >> *.csv file,and don't know the type of every column,in order to universal, i >> want to write a progrom, if i want to load and import a *.csv,i only call my >> program.do you think it feasible and what do i do? You must know the number of column horizontally. You can make tables of strings, obviously it will limit you in your options. Stefan

1 0

[Monetdb-developers] about import *.csv
by kun ren 12 Jul '10

12 Jul '10

Dear all, I want to import *.csv into MonetDB,what interface do i will use？ Best regards

2 1

Re: [Monetdb-developers] MonetDB: Jun2010 - ignore varying address in error message
by Fabian Groffen 10 Jul '10

10 Jul '10

On 10-07-2010 09:00:21 +0200, Stefan Manegold wrote: > Changeset: f5cb2db5f58f for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=f5cb2db5f58f > Modified Files: > testing/src/Mfilter.py.in > Branch: Jun2010 > Log Message: > > ignore varying address in error message > of test for bug 2547 > > + r'^(ERROR = !sql_stmt symbol$)(.*)($->token = CREATE_SCHEMA)\n', # 26: 3 Stefan, What's the point of this? The error message obviously should disappear, so it doesn't matter it differs every time. (unless someone (perhaps even me) accidentially approved this as the correct output)

2 3

[Monetdb-developers] [MonetDB-users] Announcement: New Jun2010 feature release of MonetDB suite
by Sjoerd Mullender 08 Jul '10

08 Jul '10

It is not June anymore, but nevertheless, the MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location has changed to <http://dev.monetdb.org/downloads/>. Please fix any bookmarks you may have. -- Sjoerd Mullender

3 5

[Monetdb-developers] core file on assertion violation?
by Mark Bucciarelli 06 Jul '10

06 Jul '10

Hi, I'm debugging an assertion failure I'm seeing w/ Feb2010: rel_optimizer.c:3039: find_index: Assertion `list_length(exps) == list_length(es)' failed. But I can't find the core file. Shouldn't there be one? Running a small C program that calls abort() as the merovingian user does create a core file. Thanks, m

2 1

Re: [Monetdb-developers] MonetDB: Feb2010 - Various small bugfixes (in pf property infere...
by Stefan Manegold 06 Jul '10

06 Jul '10

Jan, are this checkin to the Feb2010 branch (changeset 1e2808a8e416) and your other checkin to the Jun2010 branch (changeset cc8b3ce4c30a) meant for these branches, only, or should they also be propagated to (merged into) the Jun2010 branch (changeset 1e2808a8e416) and default branch (both changesets). In either case, do you take care of the respective merges (both doing and preventing the propagation require merging the heads) yourself, or do you prefer us to do them? Thanks! Stefan On Tue, Jul 06, 2010 at 03:32:22PM +0200, Jan Rittinger wrote: > Changeset: 1e2808a8e416 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=1e2808a8e416 > Modified Files: > pathfinder/compiler/algebra/opt/opt_complex.c > pathfinder/compiler/algebra/prop/prop_dom.c > pathfinder/compiler/sql/lalg2sql.brg > Branch: Feb2010 > Log Message: > > Various small bugfixes (in pf property inference, optimizations & SQL code generation). > > > diffs (63 lines): > > diff -r a7a0e0845e56 -r 1e2808a8e416 pathfinder/compiler/algebra/opt/opt_complex.c > --- a/pathfinder/compiler/algebra/opt/opt_complex.c Tue Jul 06 07:29:54 2010 +0200 > +++ b/pathfinder/compiler/algebra/opt/opt_complex.c Tue Jul 06 15:32:06 2010 +0200 > @@ -2118,20 +2118,21 @@ > for (unsigned int i = 0; i < schema.count; i++) > schema.items[i].name = > PFord_order_col_at (p->sem.sort.sortby, i); > - if (PFprop_ckey (p->prop, schema)) { > - *p = *PFla_rowrank ( > - L(p), > - p->sem.sort.res, > - p->sem.sort.sortby); > - modified = true; > - break; > - } > + if (PFprop_ckey (p->prop, schema)) { > + *p = *PFla_rowrank ( > + L(p), > + p->sem.sort.res, > + p->sem.sort.sortby); > + modified = true; > + break; > + } > } > > /* Replace the rownumber operator by a projection > if only its value distribution (keys) are required > instead of its real values. */ > if (!PFprop_req_value_col (p->prop, p->sem.sort.res) && > + !p->sem.sort.part && > PFord_count (p->sem.sort.sortby) == 1 && > PFprop_key (p->prop, > PFord_order_col_at (p->sem.sort.sortby, 0))) { > diff -r a7a0e0845e56 -r 1e2808a8e416 pathfinder/compiler/algebra/prop/prop_dom.c > --- a/pathfinder/compiler/algebra/prop/prop_dom.c Tue Jul 06 07:29:54 2010 +0200 > +++ b/pathfinder/compiler/algebra/prop/prop_dom.c Tue Jul 06 15:32:06 2010 +0200 > @@ -907,9 +907,11 @@ > L(n)->schema.items[i].name), > dom); > > - add_disjdom (dom, > - PFprop_dom (R(n)->prop, > - L(n)->schema.items[i].name)); > + /* we can only infer the disjointness for a single column */ > + if (n->schema.count == 1) > + add_disjdom (dom, > + PFprop_dom (R(n)->prop, > + L(n)->schema.items[i].name)); > } > break; > > diff -r a7a0e0845e56 -r 1e2808a8e416 pathfinder/compiler/sql/lalg2sql.brg > --- a/pathfinder/compiler/sql/lalg2sql.brg Tue Jul 06 07:29:54 2010 +0200 > +++ b/pathfinder/compiler/sql/lalg2sql.brg Tue Jul 06 15:32:06 2010 +0200 > @@ -4078,7 +4078,7 @@ > } > /* In case the column did not exist in > the column list already ... */ > - if (!found) > + if (!found && !IS_LITERAL(expr)) > /* ... extend the group by list. */ > groupbylist = column_list (groupbylist, expr); > } break; > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

Re: [Monetdb-developers] MonetDB: Jun2010 - undo of changeset 36556:2b71eda5b9e9 "re-appr...
by Stefan Manegold 06 Jul '10

06 Jul '10

On Tue, Jul 06, 2010 at 08:00:58AM +0200, Stefan Manegold wrote: > Changeset: 81cd1d637c34 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=81cd1d637c34 > Modified Files: > sql/src/test/leaks/Tests/check0.stable.out > sql/src/test/leaks/Tests/check1.stable.out > sql/src/test/leaks/Tests/check2.stable.out > sql/src/test/leaks/Tests/check3.stable.out > sql/src/test/leaks/Tests/check4.stable.out > sql/src/test/leaks/Tests/check5.stable.out > sql/src/test/leaks/Tests/drop3.stable.out > sql/src/test/leaks/Tests/select1.stable.out > sql/src/test/leaks/Tests/select2.stable.out > sql/src/test/leaks/Tests/temp1.stable.out > sql/src/test/leaks/Tests/temp2.stable.out > sql/src/test/leaks/Tests/temp3.stable.out > Branch: Jun2010 > Log Message: > > undo of changeset 36556:2b71eda5b9e9 "re-approved stable output --- hope this is OK and not a leak ..." > > for a yet unknown reason, tests seem to behave differently with my Jun2010 build > than with nightly testing ans Niels' build ... > The reason turned out to be a compiled & installed geom, which is nowadays of course loaded automatically if available ... Might indeed be more MonetDB5 & sql tests that behave differently depending on whether geom is available or not, and likewise MonetDB5 tests that behave differently depending on whether sql is available or not. Until we make those tests behave identically in either case, nightly testing should be the reference point. With nightly testing, when testing MonetDB5 neither sql nor geom are available, and when testing sql, geom is not available. Hence, these outputs should be approved. Stefan > > diffs (162 lines): > > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check0.stable.out > --- a/sql/src/test/leaks/Tests/check0.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check0.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 161 ] > +[ "transient", 157 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check1.stable.out > --- a/sql/src/test/leaks/Tests/check1.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check1.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 161 ] > +[ "transient", 157 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check2.stable.out > --- a/sql/src/test/leaks/Tests/check2.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check2.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 161 ] > +[ "transient", 157 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check3.stable.out > --- a/sql/src/test/leaks/Tests/check3.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check3.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 181 ] > +[ "transient", 177 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check4.stable.out > --- a/sql/src/test/leaks/Tests/check4.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check4.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 181 ] > +[ "transient", 177 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/check5.stable.out > --- a/sql/src/test/leaks/Tests/check5.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/check5.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -163,7 +163,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 181 ] > +[ "transient", 177 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/drop3.stable.out > --- a/sql/src/test/leaks/Tests/drop3.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/drop3.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -29,7 +29,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 181 ] > +[ "transient", 177 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/select1.stable.out > --- a/sql/src/test/leaks/Tests/select1.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/select1.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -34,7 +34,7 @@ > % L1 # name > % wrd # type > % 3 # length > -[ 122 ] > +[ 118 ] > #select 1; > % . # table_name > % single_value # name > @@ -45,7 +45,7 @@ > % L1 # name > % wrd # type > % 3 # length > -[ 122 ] > +[ 118 ] > > # 22:03:05 > > # 22:03:05 > Done. > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/select2.stable.out > --- a/sql/src/test/leaks/Tests/select2.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/select2.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -34,7 +34,7 @@ > % L1 # name > % wrd # type > % 3 # length > -[ 122 ] > +[ 118 ] > #select 1; > % . # table_name > % single_value # name > @@ -45,7 +45,7 @@ > % L1 # name > % wrd # type > % 3 # length > -[ 122 ] > +[ 118 ] > > # 22:03:05 > > # 22:03:05 > Done. > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/temp1.stable.out > --- a/sql/src/test/leaks/Tests/temp1.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/temp1.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -29,7 +29,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 164 ] > +[ "transient", 160 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/temp2.stable.out > --- a/sql/src/test/leaks/Tests/temp2.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/temp2.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -29,7 +29,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 164 ] > +[ "transient", 160 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > diff -r b1df4a254d3e -r 81cd1d637c34 sql/src/test/leaks/Tests/temp3.stable.out > --- a/sql/src/test/leaks/Tests/temp3.stable.out Tue Jul 06 07:31:15 2010 +0200 > +++ b/sql/src/test/leaks/Tests/temp3.stable.out Tue Jul 06 08:00:52 2010 +0200 > @@ -29,7 +29,7 @@ > % L1, L2 # name > % char, wrd # type > % 9, 3 # length > -[ "transient", 189 ] > +[ "transient", 185 ] > #select 'persistent', count(*) from bbp() as bbp where kind like 'pers%'; > % .L3, .bbp # table_name > % L3, L4 # name > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

Re: [Monetdb-developers] MonetDB: Jun2010 - added test for fixed bug 2552 PREPARE: limit ...
by Fabian Groffen 02 Jul '10

02 Jul '10

On 02-07-2010 08:48:18 +0200, Niels Nes wrote: > Changeset: 64acf9744dca for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=64acf9744dca > Modified Files: > clients/src/mapiclient/dump.c ^^^^^ > sql/src/test/BugTracker-2010/Tests/All > sql/src/test/BugTracker-2010/Tests/limit_in_prepare.Bug-2552.sql > sql/src/test/BugTracker-2010/Tests/limit_in_prepare.Bug-2552.stable.err > sql/src/test/BugTracker-2010/Tests/limit_in_prepare.Bug-2552.stable.out > Branch: Jun2010 > Log Message: > > added test for fixed bug 2552 PREPARE: limit statement not recognised > > > diffs (158 lines): > > diff -r 1ee337e08430 -r 64acf9744dca clients/src/mapiclient/dump.c > --- a/clients/src/mapiclient/dump.c Fri Jul 02 08:30:13 2010 +0200 > +++ b/clients/src/mapiclient/dump.c Fri Jul 02 08:47:25 2010 +0200 > @@ -911,7 +911,7 @@ > const char functions[] = "SELECT \"f\".\"func\" " > "FROM \"sys\".\"schemas\" \"s\"," > "\"sys\".\"functions\" \"f\" " > - "WHERE \"f\".\"sql\" = TRUE AND " > + "WHERE \"f\".\"id\" > 2000 AND " Was this an intended commit? (I also have problems with 'hg commit' doing a global commit instead of only the subtree from where you commit ...) > "\"s\".\"id\" = \"f\".\"schema_id\"" > "%s%s%s " > "ORDER BY \"f\".\"id\""; > diff -r 1ee337e08430 -r 64acf9744dca sql/src/test/BugTracker-2010/Tests/All > --- a/sql/src/test/BugTracker-2010/Tests/All Fri Jul 02 08:30:13 2010 +0200 > +++ b/sql/src/test/BugTracker-2010/Tests/All Fri Jul 02 08:47:25 2010 +0200

2 1

[Monetdb-developers] python adapter: transaction semantics and connection.close()
by Mark Bucciarelli 30 Jun '10

30 Jun '10

Hey Gijs, I'm seeing a difference in how MonetDB and SQLite handle transactions inside Django. Here's my current unit test in Django (it passes w/ SQLite and fails with MonetDB): 1 : s1 = Simple(name="mark") 2 : s1.save() 3 : 4 : s2 = Simple(name="mark") 5 : try: 6 : s2.save() 7 : except IntegrityError: 8 : pass 9 : 10 : s = Simple.objects.get(name="mark") 11 : print s What happens under SQLite is the save on line 6 fails, but the select on line 10 works. In MonetDB, at line 10, I get the error current transaction is aborted (please ROLLBACK) Here's the trace of mapi2 commands, starting from line 2 mapi2.py, cmd, operation = sINSERT INTO "testapp_simple" ("name") VALUES ('mark'); mapi2.py, cmd, operation = Xreply_size 1 mapi2.py, cmd, operation = sCOMMIT; mapi2.py, cmd, operation = Xreply_size 1000 mapi2.py, cmd, operation = sINSERT INTO "testapp_simple" ("name") VALUES ('mark'); mapi2.py, cmd, operation = Xreply_size 1000 mapi2.py, cmd, operation = sSELECT "testapp_simple"."id", "testapp_simple"."name" FROM "testapp_simple" WHERE "testapp_simple"."name" = 'mark' ; mapi2.py, cmd, operation = Xreply_size 1 mapi2.py, cmd, operation = sROLLBACK; It looks like the ROLLBACK isn't executed until the connection closes. With auto commit off, shouldn't the ROLLBACK fire as soon as the insert fails? If so, how do I fix this? Thanks m

1 1

[Monetdb-developers] python adapter: registering new Monetizer types?
by Mark Bucciarelli 30 Jun '10

30 Jun '10

Hi, I'm working on the Django adapter again, and have run into a snag. I think the Python MonetDB wrapper may need a new method, but before building a patch I wanted to share the idea and get your feedback. The issue is this: Django uses the field type SafeUnicode. Monetizer doesn't have a mapping for this, and thus raises a ProgrammerError exception. (This is unicode that is safe to disply on an HTML page, and is used for all slug fields, which are common in Django.) The SQLite adapter handles this via a register_adapter method on the Database object. PostgreSQL uses psycopg2.extensions.register_adapter(). MySQL Python adapter exposes a encoders() method on the connection object. It seems the cleanest solution would be to mimic SQLite (if you want this for one connection, you most likely want it for all connections), and be able to do something like this: Database.register_adapter( type(SafeUnicode), lambda s:unicode(s) ) But it wasn't clear to me how to implement this. Somehow that call needs to make it into Montizer mapping dictionary. Thanks, Mark

2 4

[Monetdb-developers] encoding of strings returned by Python client
by Mark Bucciarelli 28 Jun '10

28 Jun '10

Any chance the Python client could be modified to return strings in Unicode instead of UTF-8? Seems like a better fit for Python; for example, that's how Python's sqlite3 module behaves. An alternative approach is to expose a set_encoding method on the connection that let's you specify the encoding of all text fields returned. If backward compatibility is an issue, the latter idea seems better even though it adds a knob. Any comments? (Is the maintainer of the Python adapter active even on this list? I didn't see many Python-specific posts in the archives.) Thanks, m

3 12

Re: [Monetdb-developers] MonetDB: Jun2010 - make it possible to use '?' in offset and limit
by Sjoerd Mullender 23 Jun '10

23 Jun '10

Niels, can you create a Changlog entry, please? On 2010-06-22 15:25, Niels Nes wrote: > Changeset: 600a35cbf6c0 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=600a35cbf6c0 > Modified Files: > sql/src/server/bin_optimizer.mx > sql/src/server/rel_bin.mx > sql/src/server/rel_schema.mx > sql/src/server/rel_select.mx > sql/src/server/sql_parser.mx > sql/src/server/sql_statement.mx > sql/src/server/sql_symbol.mx > sql/src/test/BugTracker-2009/Tests/POWER_vs_prod.SF-2596114.stable.out > sql/src/test/Dependencies/Tests/Dependencies.stable.out > Branch: Jun2010 > Log Message: > > make it possible to use '?' in offset and limit > (transplanted from 61c37c0d76f8ee3359bda98cc25e750b5201ae56) > > > diffs (truncated from 482 to 300 lines): > > diff -r d83c56c4bba7 -r 600a35cbf6c0 sql/src/server/bin_optimizer.mx > --- a/sql/src/server/bin_optimizer.mx Tue Jun 22 14:39:09 2010 +0200 > +++ b/sql/src/server/bin_optimizer.mx Mon Jun 21 23:35:09 2010 +0200 > @@ -288,9 +288,9 @@ > stmt *r = stmt_dup(j->op2.stval); > > l = stmt_limit(l, > - s->op2.stval->op1.aval->data.val.wval, > - s->op3.stval->op1.aval->data.val.wval, > - s->flag); > + stmt_dup(s->op2.stval), > + stmt_dup(s->op3.stval), > + s->flag); > s = stmt_join(l, r, cmp_equal); > ns = bin_optimizer(c, s); > stmt_destroy(s); > @@ -311,9 +311,9 @@ > stmt *r = stmt_dup(j->op2.stval); > > r = stmt_limit(r, > - s->op2.stval->op1.aval->data.val.wval, > - s->op3.stval->op1.aval->data.val.wval, > - s->flag); > + stmt_dup(s->op2.stval), > + stmt_dup(s->op3.stval), > + s->flag); > s = stmt_join(l, r, cmp_equal); > ns = bin_optimizer(c, s); > stmt_destroy(s); > @@ -325,8 +325,8 @@ > /* try to push the limit through the order */ > if (j->type == st_order) { > s = stmt_order(stmt_limit(stmt_dup(j->op1.stval), > - s->op2.stval->op1.aval->data.val.wval, > - s->op3.stval->op1.aval->data.val.wval, > + stmt_dup(s->op2.stval), > + stmt_dup(s->op3.stval), > LIMIT_DIRECTION(j->flag,1,0)), j->flag); > ns = bin_optimizer(c, s); > stmt_destroy(s); > @@ -338,8 +338,8 @@ > /* try to push the limit through the reverse */ > if (!s->flag && j->type == st_reverse) { > s = stmt_reverse(stmt_limit(stmt_dup(j->op1.stval), > - s->op2.stval->op1.aval->data.val.wval, > - s->op3.stval->op1.aval->data.val.wval, > + stmt_dup(s->op2.stval), > + stmt_dup(s->op3.stval), > 0)); > ns = bin_optimizer(c, s); > stmt_destroy(s); > @@ -351,8 +351,9 @@ > /* try to push the limit through the mark (only if there is no offset) */ > if (!s->op2.stval->op1.aval->data.val.wval && j->type == st_mark) { > s = stmt_mark_tail(stmt_limit(stmt_dup(j->op1.stval), > - s->op2.stval->op1.aval->data.val.wval, > - s->op3.stval->op1.aval->data.val.wval, s->flag), > + stmt_dup(s->op2.stval), > + stmt_dup(s->op3.stval), > + s->flag), > j->op2.stval->op1.aval->data.val.ival); > ns = bin_optimizer(c, s); > stmt_destroy(s); > diff -r d83c56c4bba7 -r 600a35cbf6c0 sql/src/server/rel_bin.mx > --- a/sql/src/server/rel_bin.mx Tue Jun 22 14:39:09 2010 +0200 > +++ b/sql/src/server/rel_bin.mx Mon Jun 21 23:35:09 2010 +0200 > @@ -1478,32 +1478,26 @@ > return stmt_list(l); > } > > -static wrd > +static sql_exp* > topn_limit( sql_rel *rel ) > { > if (rel->exps) { > sql_exp *limit = rel->exps->h->data; > - atom *a = limit->l; > - > - return a->data.val.wval; > + > + return limit; > } > - return -1; > + return NULL; > } > > -static wrd > +static sql_exp* > topn_offset( sql_rel *rel ) > { > - wrd o = 0; > - > if (rel->exps && list_length(rel->exps) > 1) { > sql_exp *offset = rel->exps->h->next->data; > - atom *a = offset->l; > - > - o = a->data.val.wval; > - if (o <= 0) > - o = 0; > + > + return offset; > } > - return o; > + return NULL; > } > > static sql_table * > @@ -1524,15 +1518,24 @@ > list *pl; > node *en, *n; > stmt *sub = NULL, *psub = NULL; > - wrd l = -1; > + stmt *l = NULL; > > if (topn) { > - wrd o = topn_offset(topn); > - l = topn_limit(topn); > - if (l < 0) /* for now only handle topn > + sql_exp *le = topn_limit(topn); > + sql_exp *oe = topn_offset(topn); > + > + if (!le) { /* for now only handle topn > including limit, ie not just offset */ > topn = NULL; > - l += o; > + } else { > + l = exp_bin(sql, le, NULL, NULL, NULL, NULL); > + if (oe) { > + sql_subtype *wrd = sql_bind_localtype("wrd"); > + sql_subfunc *add = sql_bind_func_result(sql->session->schema, "sql_add", wrd, wrd, wrd); > + stmt *o = exp_bin(sql, oe, NULL, NULL, NULL, NULL); > + l = stmt_binop(l, o, add); > + } > + } > } > > if (!rel->exps) > @@ -1591,7 +1594,7 @@ > stmt_destroy(sub); > return NULL; > } > - limit = stmt_limit(orderbycols, 0, l, LIMIT_DIRECTION(is_ascending(orderbycole), 1, before_project)); > + limit = stmt_limit(orderbycols, stmt_atom_wrd(0), l, LIMIT_DIRECTION(is_ascending(orderbycole), 1, before_project)); > for ( n=pl->h ; n; n = n->next) { > list_append(npl, > stmt_semijoin(column(stmt_dup(n->data)), > @@ -1902,8 +1905,8 @@ > rel2bin_topn( mvc *sql, sql_rel *rel, list *refs) > { > list *newl; > - wrd l = -1, o = 0; > - stmt *sub = NULL, *order = NULL; > + sql_exp *oe = NULL, *le = NULL; > + stmt *sub = NULL, *order = NULL, *l = NULL, *o = NULL; > node *n; > > if (rel->l) { /* first construct the sub relation */ > @@ -1918,8 +1921,8 @@ > if (!sub) > return NULL; > > - l = topn_limit(rel); > - o = topn_offset(rel); > + le = topn_limit(rel); > + oe = topn_offset(rel); > > if (sub->type == st_ordered) { > stmt *s = stmt_dup(sub->op2.stval); > @@ -1932,18 +1935,25 @@ > > if (n) { > stmt *limit = NULL, *p, *j; > - wrd lmt = l; > - > - if (l < 0) { > + sql_exp *lmt = le; > + > + if (le) > + l = exp_bin(sql, le, NULL, NULL, NULL, NULL); > + if (oe) > + o = exp_bin(sql, oe, NULL, NULL, NULL, NULL); > + > + if (!le) { > l = o; > - o = 0; > + o = stmt_atom_wrd(0); > } > + if (!o) > + o = stmt_atom_wrd(0); > > if (order) { > limit = stmt_limit(stmt_dup(order), o, l, 0); > } else { > limit = stmt_limit(column(stmt_dup(n->data)), o, l, 0); > - if (lmt >= 0) { > + if (lmt) { > n = n->next; > list_append(newl, limit); > } > @@ -1953,7 +1963,7 @@ > j = find_projection_join(limit); > if (j && 0) { > p = find_pivot(j); > - if (lmt < 0) > + if (!lmt) > p = stmt_diff(stmt_dup(p), stmt_dup(limit)); > else > p = stmt_semijoin(stmt_dup(p), stmt_dup(limit)); > @@ -1971,7 +1981,7 @@ > for ( ; n; n = n->next) { > stmt *s; > > - if (lmt < 0) > + if (!lmt) > s = stmt_diff(column(stmt_dup(n->data)), stmt_dup(limit)); > else > s = stmt_semijoin(column(stmt_dup(n->data)), stmt_dup(limit)); > @@ -1979,7 +1989,7 @@ > } > } > if (order) { > - if (lmt < 0) { > + if (!lmt) { > order = stmt_diff(order, limit); > } else { > stmt_destroy(order); > diff -r d83c56c4bba7 -r 600a35cbf6c0 sql/src/server/rel_schema.mx > --- a/sql/src/server/rel_schema.mx Tue Jun 22 14:39:09 2010 +0200 > +++ b/sql/src/server/rel_schema.mx Mon Jun 21 23:35:09 2010 +0200 > @@ -859,7 +859,7 @@ > if (query->token == SQL_SELECT) { > SelectNode *sn = (SelectNode *) query; > > - if (sn->limit >= 0) > + if (sn->limit) > return sql_error(sql, 01, "CREATE VIEW: LIMIT not supported"); > if (sn->orderby) > return sql_error(sql, 01, "CREATE VIEW: ORDER BY not supported"); > diff -r d83c56c4bba7 -r 600a35cbf6c0 sql/src/server/rel_select.mx > --- a/sql/src/server/rel_select.mx Tue Jun 22 14:39:09 2010 +0200 > +++ b/sql/src/server/rel_select.mx Mon Jun 21 23:35:09 2010 +0200 > @@ -4644,12 +4644,31 @@ > if (!rel) > return NULL; > > - if (sn->limit > 0 || sn->offset > 0) { > + if (sn->limit || sn->offset) { > + sql_subtype *wrd = sql_bind_localtype("wrd"); > list *exps = new_exp_list(); > > - append(exps, exp_atom_wrd(sn->limit)); > - if (sn->offset > 0) > - append(exps, exp_atom_wrd(sn->offset)); > + if (sn->limit) { > + sql_exp *l = rel_value_exp( sql, NULL, sn->limit, 0, ek); > + > + if (!l || !(l=rel_check_type(sql, wrd, l, type_equal))) > + return NULL; > + if ((ek.card != card_relation && sn->limit) && > + (ek.card == card_value && sn->limit)) { > + sql_subaggr *zero_or_one = sql_bind_aggr(sql->session->schema, "zero_or_one", exp_subtype(l)); > + > + l = exp_aggr1(l, zero_or_one, 0, 0, CARD_ATOM, 0); > + } > + // return sql_error(sql, 01, "SELECT: LIMIT only allowed on outermost SELECT"); > + append(exps, l); > + } else > + append(exps, NULL); > + if (sn->offset) { > + sql_exp *o = rel_value_exp( sql, NULL, sn->offset, 0, ek); > + if (!o || !(o=rel_check_type(sql, wrd, o, type_equal))) > + return NULL; > + append(exps, o); > + } > rel = rel_topn(rel, exps); > } > return rel; > @@ -4670,10 +4689,6 @@ > return table_ref(sql, rel, sq); > assert(sn->s.token == SQL_SELECT); > > - if ((ek.card != card_relation && sn->limit >= 0) && > - (ek.card == card_value && sn->limit != 1)) > - return sql_error(sql, 01, "SELECT: LIMIT only allowed on outermost SELECT"); > - > if (ek.card != card_relation && sn->orderby) > return sql_error(sql, 01, "SELECT: ORDER BY only allowed on outermost SELECT"); > > diff -r d83c56c4bba7 -r 600a35cbf6c0 sql/src/server/sql_parser.mx > --- a/sql/src/server/sql_parser.mx Tue Jun 22 14:39:09 2010 +0200 > +++ b/sql/src/server/sql_parser.mx Mon Jun 21 23:35:09 2010 +0200 > @@ -388,6 +388,9 @@ > ordering_spec > simple_table > table_ref > + opt_limit > + opt_offset > + param > case_exp > case_scalar_exp > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

1 0

[Monetdb-developers] specifying NULL when using ODBC param_query
by kss ram 20 Jun '10

20 Jun '10

Hello Everyone, Is there any way to specify a null value when using ODBC param_query, in case of SQL_Query we do not really end up facing this problem as we build our own query string and then execute sql_query, sql_query is potentially unsafe of sql injection attacks, and thus using param_query makes alot of sense, but param query makes it hard to specify the data type nuetral NULL value, if we try specifying blank values like 0 in case of number and "" in case of string, these blank values will end up taking space in the database and thus would be expensive in terms of memory consumption, a database specific NULL value would leave the field null with no memory consumption. Thank you everyone in advance. kss ram

2 1

Re: [Monetdb-developers] MonetDB: default - MAP_ANONYMOUS instead of MAP_ANON
by Stefan Manegold 17 Jun '10

17 Jun '10

Niels, is this just precaution --- `man mmap` says " MAP_ANON Synonym for MAP_ANONYMOUS. Deprecated. " --- or bug fix that should be back-ported to the Jun2010 branch (I could do that)? Thanks. Stefan On Thu, Jun 17, 2010 at 09:47:40PM +0200, Niels Nes wrote: > Changeset: 57878a477b7c for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=57878a477b7c > Modified Files: > MonetDB/src/gdk/gdk_posix.mx > Branch: default > Log Message: > > MAP_ANONYMOUS instead of MAP_ANON > > > diffs (14 lines): > > diff -r 99d16aff6ce6 -r 57878a477b7c MonetDB/src/gdk/gdk_posix.mx > --- a/MonetDB/src/gdk/gdk_posix.mx Thu Jun 17 14:39:08 2010 +0200 > +++ b/MonetDB/src/gdk/gdk_posix.mx Thu Jun 17 21:47:00 2010 +0200 > @@ -2105,8 +2105,8 @@ > > #define MT_PAGESIZE(s) ((((s)-1)/MT_pagesize()+1)*MT_pagesize()) > > -#if defined(MAP_ANON) > -#define MMAP_FLAGS(f) f|MAP_ANON > +#if defined(MAP_ANONYMOUS) > +#define MMAP_FLAGS(f) f|MAP_ANONYMOUS > #define MMAP_FD -1 > #define MMAP_OPEN_DEV_ZERO int fd = 1 > #define MMAP_CLOSE_DEV_ZERO (void)fd > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] Parallel insertions in monetdb table - some records do not insert
by kss ram 16 Jun '10

16 Jun '10

Hello Everyone! Brief Overview of the issue 1. Sample Data that erlang program tries to insert in monetdb {data, [{"a","string"}, {"b","string"}], %% a,b are column names [ [{"a","a1"}, {"b","b1"}], %% key, value pair of the data - {columnname, data} [{"a","a2"}, {"b","b2"}], [{"a","a3"}, {"b","b3"}], [{"a","a4"}, {"b","b4"}], [{"a","a5"}, {"b","b5"}], [{"a","a6"}, {"b","b6"}], [{"a","a7"}, {"b","b7"}], [{"a","a8"}, {"b","b8"}], [{"a","a9"}, {"b","b9"}], [{"a","a10"}, {"b","b10"}], [{"a","a11"}, {"b","b11"}] ], {"no_of_records","12"} } 2. no of threads/processes that would do parallel inserts would be determined by the thread size i.e how many of records the thread would insert, if say 5 records then 3 threads, if 3 records then 4 threads. 3. in order to connect to monetdb from erlang, the monetdb ODBC driver is specified in ODBCConfig utility of ubuntu, driver file is /usr/lib/libMonetODBC.so 4. threads are successfully created and the ODBC module quits successfully after inserting in the erlang program, but when we look in the monetdb table, some of the rows get missed, for example row from 1 to 11 get inserted without row 3 or 4 getting inserted. could this be a problem with the ODBC driver?, am I missing any configuration?, any help would be deeply appreciated. Thank you, Ram.

3 2

Re: [Monetdb-developers] MonetDB: default - fix compilation failure opt_mapreduce.mx:141:...
by Sjoerd Mullender 10 Jun '10

10 Jun '10

Why not fix this "properly" by changing the type of cnt (and the function return value and several other variables)? At the very least you could have an assert to check that the value fits in an int. On 2010-06-10 10:54, Fabian Groffen wrote: > Changeset: 820a7637cf5e for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=820a7637cf5e > Modified Files: > MonetDB5/src/optimizer/opt_mapreduce.mx > Branch: default > Log Message: > > fix compilation failure opt_mapreduce.mx:141: '=' : conversion from 'size_t' to 'int', possible loss of data > > > diffs (12 lines): > > diff -r c7d1cd0d0bbb -r 820a7637cf5e MonetDB5/src/optimizer/opt_mapreduce.mx > --- a/MonetDB5/src/optimizer/opt_mapreduce.mx Wed Jun 09 15:30:45 2010 +0200 > +++ b/MonetDB5/src/optimizer/opt_mapreduce.mx Thu Jun 10 10:54:23 2010 +0200 > @@ -138,7 +138,7 @@ > return 0; > } > cloud = BATdescriptor(bid); > - cnt = BATcount(cloud); > + cnt = (int)BATcount(cloud); > BBPreleaseref(bid); /* we're done with it */ > return(cnt); > } > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] MonetDB: default - Add test to prevent warning
by Stefan Manegold 01 Jun '10

01 Jun '10

Arjen, is this a fix that should also be include in the upcoming Jun2010 release? If so, it would need to be back-ported to the Jun2010 branch ... Stefan On Tue, Jun 01, 2010 at 11:40:20AM +0200, Arjen de Rijke wrote: > Changeset: 74ee41649425 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=74ee41649425 > Modified Files: > clients/src/php/native/lib/php_mapi.inc > Branch: default > Log Message: > > Add test to prevent warning > > > diffs (15 lines): > > diff -r 80afe14ca780 -r 74ee41649425 clients/src/php/native/lib/php_mapi.inc > --- a/clients/src/php/native/lib/php_mapi.inc Tue Jun 01 10:02:37 2010 +0200 > +++ b/clients/src/php/native/lib/php_mapi.inc Tue Jun 01 11:37:25 2010 +0200 > @@ -768,8 +768,10 @@ > global $connection_pool; > > $connections = array(); > - foreach($connection_pool as $conn) { > + if ($connection_pool !== null) { > + foreach($connection_pool as $conn) { > $connections[] = $conn["id"]; > + } > } > > $id = hash("sha1", time()); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 1

[Monetdb-developers] mx tool status and future
by Alex Dubov 01 Jun '10

01 Jun '10

Greetings. I understand, that this issue was probably discussed before, but I couldn't quite find a way to search the list archive. It appears to me, that mx version 3, found on savannah.nongnu.org, is somewhat out of sync with the one used for monetdb (@mal and @include tags are not supported by it; adding the former was trivial, while I've set the later to no-op, as included *.mx files are nowhere to be found in the monetdb source tree). The mx doc formatter itself is also quite underdeveloped, compared to what we came to expect from systems like doxygen or sphinx. In light of the above, I wonder, are there any plans to switch to a different code documentation system (doxygen?) or is the mx tool intended for future development?

4 3

Re: [Monetdb-developers] MonetDB: default - Remove the C flag -Wno-uninitialized for Mac ...
by Stefan Manegold 21 May '10

21 May '10

Good point. Thanks! I don't expect any problems with this change, mostly because we don't have any MacOS X machine for testing, anymore; only one for buildbot --- once that's running again,we'll see ... Assuming that compilation works for you on a "modern" MacOS X (Snow Leopard), I assume that the mentioned bison.simple problems were related/specific to an older version of MacOS X, only. (MacOS X was never a primary develoment platform for us, that why disabling -Wuninitialized (by enabling -Wno-uninitialized) on MacOS X used to be - "acceptable" for us; obviously, with you using MacOS X as a primary development platform, -Wno-uninitialized should indeed not be used - (gloablly) ...) Stefan On Thu, May 20, 2010 at 06:20:02PM +0200, Jan Rittinger wrote: > Changeset: 892fc9b323aa for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=892fc9b323aa > Modified Files: > buildtools/conf/MonetDB.m4 > Branch: default > Log Message: > > Remove the C flag -Wno-uninitialized for Mac operating systems. > This helps to detect uninitialized variables on OS X. > > (Please undo the fix in case it breaks anything.) > > > diffs (12 lines): > > diff -r b181da69e3ba -r 892fc9b323aa buildtools/conf/MonetDB.m4 > --- a/buildtools/conf/MonetDB.m4 Thu May 20 17:50:15 2010 +0200 > +++ b/buildtools/conf/MonetDB.m4 Thu May 20 18:18:33 2010 +0200 > @@ -811,7 +811,7 @@ > dnl generates "X_CFLAGS =" in the generated Makefile. > NO_X_CFLAGS='X_CFLAGS' > ;; > - *-solaris*|*-darwin*|*-aix*) > + *-solaris*|*-aix*) > dnl In some cases, there is a (possibly) uninitialized > dnl variable in bison.simple ... |-( > X_CFLAGS="$X_CFLAGS -Wno-uninitialized" > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] MonetDB/SQL: bulk-loading large data sets & adding key constraints
by Stefan Manegold 15 May '10

15 May '10

Dear all, derived from recent experiments and intensive testing, here are some recommendations for bulk-loading large data sets (in particular those larger than your available main memory) in MonetDB/SQL and adding constraints, i.e., primary and foreign keys. First of all, bulk loading should be done from (possibly gzip or bzip2 compressed) CSV files using the SQL `COPY INTO` command; cf., http://monetdb.cwi.nl/SQL/Documentation/Data-Manipulation.html In case your tables come with primary or foreign key constraints, it is recommended to omit these constrains when initially creating the tables. Then bulk-load (`COPY INTO`) your data. Once the data is loaded add the key constraints one by one using the SQL `ALTER TABLE ... ADD CONSTRAINT ...` command. Note, though, that the current Feb2010 release of MonetDB/SQL might experience some performance problems when adding constraints on data sets larger than main memory on multi-core machines. The reason it a non-optimal query translation and plan parallelization. For the time being, i.e., until we manage to fix this problem properly, it is recommended to resort to the optimizer pipeline of the Nov2009 release (for insiders, this effectively means disabling the "mitosis" optimizer), (only) while adding the key constraints. To do so, simply activate the Nov2009 optimizer pipeline via SET optimizer='nov2009_pipe'; just before adding the key constraints via `ALTER TABLE ... ADD CONSTRAINTS` and then restore the default optimizer pipeline via SET optimizer='default_pipe'; once you're done with adding constraints. For TPCH scale factor 100 (100 GB) on a 8 GB 4-core machine, this reduces the total time to add all key constraints from ~1.5h to ~1h. For SSB scale factor 100 (100 GB) on a 8 GB 4-core machine, this reduces the total time to add all key constraints from ~4h to ~1h. Hope this helps. Please don't hesitate to comment or ask questions. We'll let you know, one we managed to fix this problem properly. Kind regards, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

Re: [Monetdb-developers] MonetDB: Jun2010 - Fix compilation on Windows
by Stefan Manegold 13 May '10

13 May '10

On Thu, May 13, 2010 at 10:02:14AM +0200, Fabian Groffen wrote: > Changeset: db9f1c910dbb for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=db9f1c910dbb > Modified Files: > MonetDB5/src/mal/mal_linker.mx > Branch: Jun2010 > Log Message: > > Fix compilation on Windows > > .\..\..\..\src\mal\mal_linker.mx(62): catastrophic error: could not open > source file "dirent.h" > > Seems Sabaoth can use struct dirent without including dirent.h as well, > so I guess some gdk header takes care of this. I just removed dirent.h > include. > FYI: MonetDB/src/gdk/gdk.mx-#ifdef HAVE_DIRENT_H MonetDB/src/gdk/gdk.mx:# include <dirent.h> MonetDB/src/gdk/gdk.mx:# define NAMLEN(dirent) strlen((dirent)->d_name) MonetDB/src/gdk/gdk.mx-#else MonetDB/src/gdk/gdk.mx:# define dirent direct MonetDB/src/gdk/gdk.mx:# define NAMLEN(dirent) (dirent)->d_namlen MonetDB/src/gdk/gdk.mx-# ifdef HAVE_SYS_NDIR_H MonetDB/src/gdk/gdk.mx-# include <sys/ndir.h> MonetDB/src/gdk/gdk.mx-# endif MonetDB/src/gdk/gdk.mx-# ifdef HAVE_SYS_DIR_H MonetDB/src/gdk/gdk.mx-# include <sys/dir.h> MonetDB/src/gdk/gdk.mx-# endif MonetDB/src/gdk/gdk.mx-# ifdef HAVE_NDIR_H MonetDB/src/gdk/gdk.mx-# include <ndir.h> MonetDB/src/gdk/gdk.mx-# endif MonetDB/src/gdk/gdk.mx-#endif Stefan > > diffs (11 lines): > > diff -r e1c42586a988 -r db9f1c910dbb MonetDB5/src/mal/mal_linker.mx > --- a/MonetDB5/src/mal/mal_linker.mx Wed May 12 16:58:04 2010 +0200 > +++ b/MonetDB5/src/mal/mal_linker.mx Thu May 13 10:02:00 2010 +0200 > @@ -59,7 +59,6 @@ > #endif > > #include <sys/types.h> /* opendir */ > -#include <dirent.h> /* opendir */ > > /* #define DEBUG_MAL_LINKER */ > #define MONET64 1 > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

Re: [Monetdb-developers] MonetDB: Jun2010 - include sql is no longer in the arguments
by Stefan Manegold 10 May '10

10 May '10

Oops ;-) Sorry, missed that ... Thanks for watching & fixing! Stefan On Mon, May 10, 2010 at 08:16:01AM +0200, Fabian Groffen wrote: > Changeset: 2716b35c7688 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2716b35c7688 > Modified Files: > sql/src/backends/monet5/merovingian/merovingian_forkmserver.c > Branch: Jun2010 > Log Message: > > include sql is no longer in the arguments > > > diffs (12 lines): > > diff -r a901acf756bb -r 2716b35c7688 sql/src/backends/monet5/merovingian/merovingian_forkmserver.c > --- a/sql/src/backends/monet5/merovingian/merovingian_forkmserver.c Mon May 10 08:06:52 2010 +0200 > +++ b/sql/src/backends/monet5/merovingian/merovingian_forkmserver.c Mon May 10 08:15:50 2010 +0200 > @@ -226,7 +226,7 @@ > str master = NULL; > str slave = NULL; > str pipeline = NULL; > - str argv[27]; /* for the exec arguments */ > + str argv[26]; /* for the exec arguments */ > confkeyval *ckv, *kv; > int c = 0; > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 1

[Monetdb-developers] About the use of hg transplant
by Fabian Groffen 09 May '10

09 May '10

For those using hg transplant to backport changes. Using the -m flag of transplant causes a merge commit to occur. This means two heads (could be branches) are merged into one. For some not yet fully understood reason this causes a hg merge to perform a large batch of changes, probably recursively performing the full changes of the branch that was transplanted from. I think for backports it is better not to use -m, because it artificially says you moved over everything that was supposed to be moved over. However, in fact you didn't. Just don't make a merge commit. I found out that this is easy to deal with in the propagation, since transplant can be used in the following fashion: % cvs transplant --log -b Jun2010 -p 456afbaaf416 -p 0ee83fdb0e97 -m 63f8c7be221a applying 63f8c7be221a 35864:63f8c7be221a merged at cfbfaf940bb2 % Which in this case means: ignore changesets 456afbaaf416 and 0ee83fdb0e97, and at changeset 63f8c7be221a pretend a merge was done, which happens to be correct. (Actually, without the -p and -m opts, transplant will ask per changeset what to do with it, so it seems better suited to do regulated propagation merges.)

2 2

Re: [Monetdb-developers] MonetDB: default - Revert the changes of May 04 to recover perfo...
by Stefan Manegold 09 May '10

09 May '10

In case the changes of May 04 indeed caused the reported performance degradation, I assume this checkin should then also be back-ported to the Jun2010 branch that was created on May 06 and hence also contains the problematic changes of May 04, right? Once confirmed and I can back-port this checkin. Stefan On Sat, May 08, 2010 at 08:30:35PM +0200, Martin.Kersten(a)cwi.nl wrote: > Changeset: e9c1206c5008 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e9c1206c5008 > Modified Files: > MonetDB5/src/mal/mal_interpreter.mx > clients/src/python/test/capabilities.py > clients/src/python/test/dbapi20.py > geom/src/sql/geom.sql > Branch: default > Log Message: > > Revert the changes of May 04 to recover performance. > > > diffs (76 lines): > > diff -r dd777c76a3f9 -r e9c1206c5008 MonetDB5/src/mal/mal_interpreter.mx > --- a/MonetDB5/src/mal/mal_interpreter.mx Sat May 08 09:16:12 2010 +0200 > +++ b/MonetDB5/src/mal/mal_interpreter.mx Sat May 08 20:30:23 2010 +0200 > @@ -341,7 +341,7 @@ > > /* pass the new debug mode to the caller */ > if (stk->cmd && env && stk->cmd!='f') env->cmd = stk->cmd; > - if ( garbageControl(getInstrPtr(mb,0)) ) > + if ( !stk->keepAlive && garbageControl(getInstrPtr(mb,0)) ) > garbageCollector(cntxt, mb,stk, env != stk); > @:endProfile(stk)@ > return ret; > @@ -361,15 +361,19 @@ > MalStkPtr stk, MalStkPtr env, InstrPtr pcicaller) > { > str ret; > + int keepAlive; > > if (stk == NULL) > throw(MAL, "mal.interpreter", MAL_STACK_FAIL); > + keepAlive = stk->keepAlive; > if(env && stk && stk->cmd!='f') stk->cmd = env->cmd ; > > ret = runMALsequence(cntxt, mb, startpc, stoppc, stk, env, pcicaller); > > /* pass the new debug mode to the caller */ > if (env && stk->cmd!='f') env->cmd = stk->cmd; > + if ( keepAlive == 0 && garbageControl(getInstrPtr(mb,0)) ) > + garbageCollector(cntxt, mb,stk,env!=stk); > return ret; > } > @- > @@ -1336,7 +1340,6 @@ > { > DataFlow flow = NULL; > str ret = MAL_SUCCEED; > - int keepalive; > > #ifdef DEBUG_FLOW > stream_printf(GDKstdout, "runMALdataflow for block %d - %d\n", startpc,stoppc); > @@ -1387,9 +1390,6 @@ > flow->stop = stoppc; > flow->mb = mb; > flow->stk = stk; > - /* avoid the garbage collection call at the end of interpreter loop */ > - keepalive = flow->stk->keepAlive; > - flow->stk->keepAlive= TRUE; > > flow->status = (char*) GDKzalloc((stoppc-startpc+1)); > flow->blocked = (char*) GDKzalloc(sizeof(char)*mb->vtop); > @@ -1397,7 +1397,6 @@ > mal_unset_lock(mal_contextLock, "runMALdataflow"); > > ret = DFLOWscheduler(flow); > - flow->stk->keepAlive= keepalive; > GDKfree(flow->status); > GDKfree(flow->blocked); > GDKfree(flow->inuse); > @@ -1764,7 +1763,7 @@ > sbackup[i]= 0; > backup[i]= 0; > garbage[i]= -1; > - if (stk->stk[getArg(pci,i)].vtype == TYPE_bat && getEndOfLife(mb,getArg(pci,i)) == stkpc && isNotUsedIn(pci,i+1,getArg(pci,i)) && stk->keepAlive == 0){ > + if (stk->stk[getArg(pci,i)].vtype == TYPE_bat && getEndOfLife(mb,getArg(pci,i)) == stkpc && isNotUsedIn(pci,i+1,getArg(pci,i)) ){ > garbage[i]= getArg(pci,i); > #ifdef DEBUG_GC > stream_printf(GDKstdout,"GC %d %s prep\n",getArg(pci,i), getArgName(mb,pci,i)); > @@ -2345,8 +2344,6 @@ > int k; > ValPtr v; > > - if ( stk->keepAlive) > - return; > #ifdef STACKTRACE > stream_printf(cntxt->fdout,"#--->stack before garbage collector\n"); > printStack(cntxt->fdout,mb,stk,0); > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

3 2

Re: [Monetdb-developers] commit mail subject
by Fabian Groffen 07 May '10

07 May '10

On 07-05-2010 18:38:42 +0200, Stefan Manegold wrote: > On Fri, May 07, 2010 at 06:31:21PM +0200, Jan Rittinger wrote: > > Hi all, > > > > I think the subject format of the checkin message mail could be improved by changing the format from 'MonetDB: <log-message> (<branch>)' or sometimes 'MonetDB: <partial log-message>...' to 'MonetDB (<branch>): <log-message>' or 'MonetDB (<branch>): <partial log-message>...'. > > FYI: > > Mercurial uses the first line of checkin messages as summary in (e.g.) logs, > etc. and as email subject. > > It depends on the committer how useful this first line (or "partial > log-message") in isolation, and hence, the summary and email subject is > --- in particular for other --- but that holds for the complete checkin > message, too ... If the first line is too long (like a full page) the line is truncated by the script, guess that's what Jan refers too. Makes sense from a mail point of view. > (Sjoerd pointed this out in his Mercurial intorduction during MADAM and in > his MonetDB-Mercirial document.) > > > In short I vote for switching the order of <log-message> and <branch> in the subject. > > Fabian patched the Mercurial setup to add the brnach to the checkin email > subject. I guess, he can also prepend instead of append it. > > Would be fine by me. (In fact, I like your suggestion ;-)) Ok, that makes the score already 2 then :)

2 3

[Monetdb-developers] commit mail subject
by Fabian Groffen 07 May '10

07 May '10

On 07-05-2010 18:31:21 +0200, Jan Rittinger wrote: > Hi all, > > I think the subject format of the checkin message mail could be improved by changing the format from 'MonetDB: <log-message> (<branch>)' or sometimes 'MonetDB: <partial log-message>...' to 'MonetDB (<branch>): <log-message>' or 'MonetDB (<branch>): <partial log-message>...'. > > In short I vote for switching the order of <log-message> and <branch> in the subject. I had that first, but it looked somewhat awkward. I have no problems with changing it back though. Other people with an opinion on this one? Also, there is a X-Hg-Branch header in the mail, in case you want to easily filter them.

1 0

[Monetdb-developers] Move of SourceForge bugtracker to MonetDB Bugzilla
by Fabian Groffen 07 May '10

07 May '10

This message is important for all that used to report bugs on our SourceForge tracker, or are planning to report bugs in the future. With the move of our CVS repository to a new Mercurial repository, we have also migrated the bugtracker from SourceForge to a Bugzilla instance. The old SourceForge trackers are no longer in use. Please DO NOT file any bugs there, as they will get lost. We have removed their presence as far as possible to avoid any confusion here. The new location to search, read and file new bugs is here: http://dev.monetdb.org/bugzilla/ (Please bear with us while we try to get the interface/skin well suited for our project.) All bugs that were previously on the SourceForge trackers have been migrated to the new Bugzilla instance. The bug numbers could not be retained, hence all have been renumbered. See [1] for dealing with this in Bugzilla. All users that have ever reported a bug, commented on a bug, or been an assignee for a bug on SourceForge have been imported in Bugzilla. This means that for all those users a <sf-nickname>@users.sourceforge.net account has been created. Exception to this are users who work at CWI, whose addresses have been mapped to their CWI address. If you want to use your migrated account, simply click the "Forgot Password" link on the Bugzilla main page. It will send an email to your (SourceForge) email address with a token to allow you to set a password. If you are a new user, you can simply sign up on our Bugzilla using the "Open a New Account" button. In both cases, make sure you look through the preferences, in particular the "Email Preferences". For migrated accounts these preferences are set to exclude sending any bug email, which may or may not be what you prefer. Feel free to ask for any help if problems arise. [1] Searching an old bug using its bugnumber: If you have an old SourceForge bug number and need to find tha associated bug in Bugzilla, open up an advanced search, and enter the old bug number in the URL field. Make sure that you select the appropriate bug resolutions (e.g. include RESOLVED) and search. Each migrated bug has an URL set to the original bug on SourceForge, hence is easily found this way.

4 6

[Monetdb-developers] New Jun2010 stable branch created
by Sjoerd Mullender 07 May '10

07 May '10

I have created new stable branches for the next feature release. This branch was created in order to stabilize the code before the feature release. From now on *only* bug fixes may go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. I also reserve the right to admit changes that are not strictly bug fixes. All packages use the same name for the branch: Jun2010. So in each of your stable checkout directories you can do hg update -rJun2010 to update to this new branch. -- Sjoerd Mullender

2 1

[Monetdb-developers] MonetDB testing temporarily suspended
by Stefan Manegold 06 May '10

06 May '10

Dear fellow developers, it appears that the directory structure used by MonetDB testing (read: the "TestTools"), in particular the setup that adminstrative data and testing results (incl. archives of the last 30 days) end up "polluting" the source tree, is not suitable for Mercurial --- at least the not totally up-to-date Mercurial 1.1 on our testing server that is still running Fedora 8 cannot cope well with the lots of extra "unknown" files and directories in the source tree. I have not yet tested, whether version of Mercurial (then running from another machine via NFS) would cope better. I guess, it's finally time to revise the design of TestTools --- at least the directory structure it uses --- after all, the priciple design is still from the last century. I will start doing so (later) today, but cannot promise that I'll finish the liberation from this legacy today. Hence, MonetDB testing (which did not run for the given reasons this night) will remain suspended until I finish the re-design and implementaion of a revised directory structure (or at least a temporary work around. I'll keep you posted. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 2

Re: [Monetdb-developers] SQL main BAT fallback creates collision
by Isidor Zeuner 05 May '10

05 May '10

> > Where can I find the new bug tracker? > > We're still working on it. We aim to get it online sometime today. > Thanks. The new tracker is working fine. > > By slightly changing the test submitted with bug #2995671, I was able > > to trigger an unrelated bug (see the attached test file .. > > I have attached a patch (no-fallback-ibid.diff) which avoids this > > problem by disabling the iBAT fallback, creating an empty main BAT > > instead. I have submitted the issue on the new tracker ([1]). I slightly changed the patch so the empty main BAT is not saved to bat->bid, as this caused inserts done inside a transaction not to be committed. Is there some documentation on the meaning of 0 == bat->bid with respect to the stored data? > > > > To improve the subset of possible interactions between components > > which is tested, have you considered a fuzz-testing-like approach? > > Best regards, Isidor [1] http://bugs.monetdb.org/show_bug.cgi?id=2529

1 0

[Monetdb-developers] IMPORTANT: move of MonetDB repository imminent
by Sjoerd Mullender 04 May '10

04 May '10

On Monday, May 3, 2010, we are going to move the MonetDB repository over from the Sourceforge-hosted CVS repository to a Mercurial repository hosted on dev.monetdb.org. On March 18 I sent a pre announcement to the monetdb-developers list (IMPORTANT: changes are afoot). [1] In particular, the following CVS modules will be converted and moved: MonetDB, MonetDB4, MonetDB5, buildtools, clients, geom, java, pathfinder, sql, template, testing. The other CVS modules that are hosted at Sourceforge (pf-haskell, pf-tools, xml) will remain there, at least for now. This means that as of Monday, the affected modules in the CVS repository will be made read only (it will remain readable for a long time, just not writable). When the move is complete, the repository will open at the following location: http://dev.monetdb.org/hg/MonetDB/ This URL can be used to browse the repository using a web browser, and it can be used as the URL to make a Mercurial clone (hg clone http://dev.monetdb.org/hg/MonetDB/) This will be a read-only version of the repository. Core developers will get read-write access at the URL ssh://hg@dev.monetdb.org/MonetDB/ I wrote a document to help with the conversion to Mercurial, both describing the way to use Mercurial, and how to convert your CVS working set to a Mercurial clone. For now (until we find a better place), this document can be accessed at: http://homepages.cwi.nl/~sjoerd/downloads/monetdb/MonetDB-Mercurial.html http://homepages.cwi.nl/~sjoerd/downloads/monetdb/MonetDB-Mercurial.pdf As part of the move, I will subscribe subscribers to the various monetdb checkin mailing lists to the new mailing list checkin-list(a)monetdb.org. If you are subscribed to one or more of monetdb-checkins(a)lists.sourceforge.net monetdb-sql-checkins(a)lists.sourceforge.net monetdb-pf-checkins(a)lists.sourcefoge.net I will subscribe you to the list checkin-list(a)monetdb.org. This I will do on Monday as well, so you should get a message to that effect on Monday. [1] http://sourceforge.net/mailarchive/forum.php?thread_name=4BA239BF.4090307%4… -- Sjoerd Mullender

3 2

[Monetdb-developers] bug report on trigger statement
by Huabing 03 May '10

03 May '10

I run following sql statements on MonetDB server v5.18.5 (32-bit), based on kernel v1.36.5 (32-bit oids) create table a (a int, b int); create table ta(a int, b int); insert into ta values(1,2); insert into ta values(3,4); insert into ta values(5,6); create trigger d_ta after delete on ta referencing old row as o for each row insert into a values(o.a, o.b); delete from ta where a=1; select * from a; drop trigger d_ta; drop table ta; drop table a; The output is: +---+-----+ | a | b | +==+==+ | 3 | 4 | | 3 | 4 | | 5 | 6 | | 5 | 6 | +---+----+ Obviously, the trigger functions wrong.

1 0

[Monetdb-developers] regarding clustering in monetdb
by Puneet Birwa 02 May '10

02 May '10

hello, i have the MonetDB-Nov2009-SuperBall-SP2.tar.bz2 file after extracting this i install the monetdb . i want to know whether the above version of monetdb which i an using, uses clustering for doing join operations. if no please tell me upto what version of monetdb clustering is used for doing join. please reply as soon as possible. thank you

1 0

[Monetdb-developers] regarding clustering in monetdb
by Puneet Birwa 02 May '10

02 May '10

hello, i have the MonetDB-Nov2009-SuperBall-SP2.tar.bz2 file after extracting this i install the monetdb . i want to know whether the above version of monetdb which i an using, uses clustering for doing join operations. if no please tell me upto what version of monetdb clustering is used for doing join. please reply as soon as possible. thank you

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal tablet_sql.mx, Feb2010, 1.44, 1.44.2.1
by Sjoerd Mullender 25 Apr '10

25 Apr '10

Should I build a new release candidate with this fix included? On 2010-04-25 12:07, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv6881/src/modules/mal > > Modified Files: > Tag: Feb2010 > tablet_sql.mx > Log Message: > fixed bug in loader, now (again) we handle embedded seperators properly > > > Index: tablet_sql.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet_sql.mx,v > retrieving revision 1.44 > retrieving revision 1.44.2.1 > diff -u -d -r1.44 -r1.44.2.1 > --- tablet_sql.mx 10 Jan 2010 14:16:36 -0000 1.44 > +++ tablet_sql.mx 25 Apr 2010 10:07:29 -0000 1.44.2.1 > @@ -448,11 +448,10 @@ > int res = 0; /* < 0: error, > 0: success, == 0: continue processing */ > int j; > BUN i; > - char q=0; > size_t rseplen; > READERtask *task= (READERtask*) GDKzalloc(sizeof(READERtask)); > READERtask ptask[16]; > - int threads= GDKnr_threads < 16? GDKnr_threads: 16; > + int threads= (!maxrow || maxrow > (1<<16)) ? (GDKnr_threads < 16? GDKnr_threads: 16):1; > lng lio =0, tio, t1 = 0, total=0, iototal=0; > int vmtrim = GDK_vm_trim; > > @@ -603,6 +602,7 @@ > rethink the matter. */ > e = s; > while (s < end && (maxrow < 0 || cnt < (BUN) maxrow) ) { > + char q=0; > /* tokenize the record completely > the format of the input should comply to > the following grammar rule > @@ -617,55 +617,50 @@ > */ > if ( quote == 0) { > if (rseplen == 1) > - for ( ; *e ; e++) > - { > + for ( ; *e ; e++) { > if ( *e == '\\' ){ > e++; > continue; > } > if (*e == *rsep ) > - break; > + break; > } > else > - for ( ; *e ; e++) > - { > + for ( ; *e ; e++) { > if ( *e == '\\' ) { > e++; > continue; > } > if (*e == *rsep && strncmp(e, rsep, rseplen) == 0) > - break; > + break; > } > if ( *e == 0) > e = 0; /* nonterminated record, we need more */ > - } else > - if ( rseplen == 1 ) { > - for ( ; *e ; e++) > - { > + } else if ( rseplen == 1 ) { > + for ( ; *e ; e++) { > if (*e == q) > - q = 0; > - else if (*e == quote) > - q = *e; > - else if (*e == '\\') { > - if (e[1]) > - e++; > - } else if (*e == *rsep ) > - break; > + q = 0; > + else if (*e == quote) > + q = *e; > + else if (*e == '\\') { > + if (e[1]) > + e++; > + } else if (!q && *e == *rsep ) > + break; > } > if ( *e == 0) > e = 0; /* nonterminated record, we need more */ > } else { > - for ( ; *e ; e++) > - { > + for ( ; *e ; e++) { > if (*e == q) > - q = 0; > - else if (*e == quote) > - q = *e; > - else if (*e == '\\') { > - if (e[1]) > - e++; > - } else if (*e == *rsep && strncmp(e, rsep, rseplen) == 0 ) > - break; > + q = 0; > + else if (*e == quote) > + q = *e; > + else if (*e == '\\') { > + if (e[1]) > + e++; > + } else if (!q && *e == *rsep && strncmp(e, rsep, rseplen) == 0 ) > + break; > } > if (*e == 0 ) > e = 0; /* nonterminated record, we need more */ > > > ------------------------------------------------------------------------------ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] MonetDB memory management
by Xiao Lei 23 Apr '10

23 Apr '10

Hi, Does anyone has the same memory management problem as me? When I inserted many records like: Insert into tbl1 (id,dt) values(1,blob ‘…………………………………………’); Memory is used up to 1G. That's OK. But when I delete the whole table like: Delete from tbl1; None memory is released, still 1G is used. And when i insert again, memory used keeps increasing. Is this a problem or is it the way MonetDB works? Thx. -- Best Regards, Xiao Lei

3 3

[Monetdb-developers] Nov2009 segmentation fault (invalid read)
by Isidor Zeuner 23 Apr '10

23 Apr '10

Dear MonetDB developers, I am currently experiencing segmentation faults on SQL-based data processing using the MonetDB Nov2009 branch. As the processing is rather complex, I don't know when I will be able to isolate a test case. Based on the clues I gathered (see below), could you give me some pointers on how to investigate further? It always crashes on this query: DELETE FROM progress WHERE EXISTS ( SELECT true FROM finalization WHERE finalization.id=progress.id AND finalization.category=active ); The related tables are defined like this: CREATE TABLE progress ( id VARCHAR(32), attempt INTEGER, clearance INTEGER, PRIMARY KEY (id) ); CREATE TABLE finalization ( category VARCHAR(32), id VARCHAR(32), governing VARCHAR(64), sub_governing VARCHAR(64), domain VARCHAR(64), reason VARCHAR(16), epoch INTEGER, size INTEGER, PRIMARY KEY (category,id,governing,sub_governing,domain,reason,epoch) ); On the crash, the stack backtrace always looks like this: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x42006940 (LWP 17625)] 0x00002b8e691fdad8 in strCmp (l=0x2aaab97c6a20 "06EED104D173", r=0xdbdbdbdbe98848f3 <Address 0xdbdbdbdbe98848f3 out of bounds>) at gdk_atoms.mx:1855 1855 return GDK_STRCMP(l, r); (gdb) bt #0 0x00002b8e691fdad8 in strCmp (l=0x2aaab97c6a20 "06EED104D173", r=0xdbdbdbdbe98848f3 <Address 0xdbdbdbdbe98848f3 out of bounds>) at gdk_atoms.mx:1855 #1 0x00002b8e693d7322 in BAThashjoin (l=0x2aaab9417188, r=0xd614ba0, estimate=9223372036854775807) at gdk_relop.mx:452 #2 0x00002b8e6940e151 in batjoin (l=0x2aaab9417188, r=0xd614ba0, estimate=9223372036854775807, swap=1 '\001') at gdk_relop.mx:1244 #3 0x00002b8e6940e193 in BATjoin (l=0x2aaab9417188, r=0xd614ba0, estimate=9223372036854775807) at gdk_relop.mx:1251 #4 0x00002aaaaafd105e in CMDjoin () from /opt/monetdb-nov2009/lib64/MonetDB5/lib/lib_algebra.so.5 #5 0x00002aaaaaffcef5 in ALGjoinestimate () from /opt/monetdb-nov2009/lib64/MonetDB5/lib/lib_algebra.so.5 #6 0x00002aaaaaffcfc1 in ALGjoin () from /opt/monetdb-nov2009/lib64/MonetDB5/lib/lib_algebra.so.5 #7 0x00002b8e6868b106 in DFLOWstep () from /opt/monetdb-nov2009/lib64/libmonetdb5.so.5 #8 0x00002b8e6868ed7d in runDFLOWworker () from /opt/monetdb-nov2009/lib64/libmonetdb5.so.5 #9 0x00000039d04064a7 in start_thread () from /lib64/libpthread.so.0 #10 0x00000039cf8d3c2d in clone () from /lib64/libc.so.6 The second argument's address to the comparison at the top has always one or more 0xdb memory deletion markers, so it looks like it is at least overlapping with deleted space. Please note that I could not create a backtrace with full symbols as the error does not occur with the MonetDB5 module being compiled with debugging enabled. Best regards, Isidor

1 1

[Monetdb-developers] Fw: BATcacheid
by anshul gangwar 22 Apr '10

22 Apr '10

Does anybody have any idea ............... hello Are BATcacheid's real column id's i.e those BATid's which are given at the time of storage of the document(static id's). If no than from where we can print those static BATid's

1 0

[Monetdb-developers] BATcacheid
by anshul gangwar 21 Apr '10

21 Apr '10

hello Are BATcacheid's real column id's i.e those BATid's which are given at the time of storage of the document(static id's). If no than from where we can print those static BATid's

1 0

[Monetdb-developers] ~~Hi~~
by Erin's Ghost 21 Apr '10

21 Apr '10

http://template09.mydevnet.ca/NepK4RsZ0U.htm

1 0

[Monetdb-developers] flow diagram
by anshul gangwar 21 Apr '10

21 Apr '10

hello Where I can find the full flow diagram of execution of monetdb. Apart from that We are printing the BATcacheid of the columns passed to the BATfetchjoin function ,We are getting the different BATCacheids . So what is the reason behind that.

2 1

[Monetdb-developers] flow diagram
by anshul gangwar 21 Apr '10

21 Apr '10

hello I was talking about flow diagram image, so that i can visualize that.

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql configure.ag, , 1.166, 1.167
by Milena Ivanova 14 Apr '10

14 Apr '10

Fabian, where should MERO_LOG be defined? Is it an env. variable? A after your last check in, I got /ufs/milena/src/sql/src/backends/monet5/merovingian/merovingian.c: In function ‘main’: /ufs/milena/src/sql/src/backends/monet5/merovingian/merovingian.c:452: error: ‘MERO_LOG’ undeclared (first use in this function) Milena On 14/04/10 11:54, Fabian wrote: > Update of /cvsroot/monetdb/sql > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv6441 > > Modified Files: > configure.ag > Log Message: > Hardwire the default location for the logs in the sources, document them appropriately > > Index: configure.ag > =================================================================== > RCS file: /cvsroot/monetdb/sql/configure.ag,v > retrieving revision 1.166 > retrieving revision 1.167 > diff -u -d -r1.166 -r1.167 > --- configure.ag 13 Apr 2010 16:39:28 -0000 1.166 > +++ configure.ag 14 Apr 2010 09:54:16 -0000 1.167 > @@ -124,6 +124,8 @@ > AC_CHECK_HEADERS([uuid/uuid.h]) > AC_SEARCH_LIBS([uuid_generate], [uuid]) > AC_SUBST([MONETDB5_CONFFILE]) > +AC_SUBST([MERO_LOG], [$MONETDB5_LOGDIR/merovingian.log]) > +AC_DEFINE_UNQUOTED([MERO_LOG], ["$MONETDB5_LOGDIR/merovingian.log"], [where Merovingian will store its logs by default]) > > dnl CFLAGS for our code are stricter than what autoconf can cope with. > CFLAGS="$CFLAGS \$(X_CFLAGS)" > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins >

1 1

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient MapiClient.mx, , 1.201, 1.202
by Stefan Manegold 14 Apr '10

14 Apr '10

On Tue, Apr 13, 2010 at 12:43:06PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv15694 > > Modified Files: > MapiClient.mx > Log Message: > Once again print timer information to stderr. > If you want to combine timer information with the regular output, you > can do > mclient -t ... 2>&1 > This works since the standard output is flushed before the timer > information is printed. > And if you only want timer information, you can do > mclient -t ... >/dev/null > I briefly tried to do the same for break-down timings given by pathfinder (MonetDB/XQuery), but the changes (see diff below) don't help, mainly since the break-down timings are printed in MIL, and even fprintf(stderr(), ...) in MIL goes to stdout due to MonetDB/src/gdk/gdk.mx:2737:#define GDKerr GDKout which is probably required because MAPI only used one stream. I'm not sure whether it's worth to persue this fruther, e.g., by making mclient aware of the pathfinder/XQuery break-down timings and have re-direct them to (its) stderr ... Stefan ======== Index: pathfinder/compiler/include/mil.h =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/include/mil.h,v retrieving revision 1.89 diff -u -r1.89 mil.h --- pathfinder/compiler/include/mil.h 7 Jan 2010 15:24:27 -0000 1.89 +++ pathfinder/compiler/include/mil.h 14 Apr 2010 10:33:26 -0000 @@ -385,6 +385,7 @@ , m_print /**< MIL print() function */ , m_printf /**< MIL printf() function */ + , m_printerr /**< MIL fprintf(stderr(),...) function */ , m_col_name /**< assign BAT column name (for debugging only) */ @@ -1014,6 +1015,7 @@ PFmil_t * PFmil_print (const PFmil_t *); PFmil_t * PFmil_printf (const PFmil_t *); +PFmil_t * PFmil_printerr (const PFmil_t *); PFmil_t * PFmil_col_name (const PFmil_t *, const PFmil_t *); PFmil_t * PFmil_comment (const char *, ...) __attribute__ ((format (printf, 1, 2))); Index: pathfinder/compiler/compile.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/compile.c,v retrieving revision 1.186 diff -u -r1.186 compile.c --- pathfinder/compiler/compile.c 7 Jan 2010 15:24:27 -0000 1.186 +++ pathfinder/compiler/compile.c 14 Apr 2010 10:33:26 -0000 @@ -801,7 +801,7 @@ /* make runtime timing available */ if (status->timing) { mroot = PFmil_seq (mroot, - PFmil_printf ( + PFmil_printerr ( PFmil_arg (PFmil_lit_str ("\nShred % 10.3f msec" "\nQuery % 10.3f msec" "\nPrint % 10.3f msec\n"), @@ -1082,7 +1082,7 @@ if (!strncmp ("timing", genType, 6)) /* add timing information */ mroot = PFmil_seq (mroot, - PFmil_printf ( + PFmil_printerr ( PFmil_arg (PFmil_lit_str ("\nTrans % 10.3f msec" "\nShred % 10.3f msec" "\nQuery % 10.3f msec" Index: pathfinder/compiler/mil/mil.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/mil.c,v retrieving revision 1.86 diff -u -r1.86 mil.c --- pathfinder/compiler/mil/mil.c 7 Jan 2010 15:24:28 -0000 1.86 +++ pathfinder/compiler/mil/mil.c 14 Apr 2010 10:33:26 -0000 @@ -1896,6 +1896,12 @@ } PFmil_t * +PFmil_printerr (const PFmil_t *args) +{ + return wire1 (m_printerr, args); +} + +PFmil_t * PFmil_col_name (const PFmil_t *bat, const PFmil_t *name) { return wire2 (m_col_name, bat, name); Index: pathfinder/compiler/mil/mil_dce.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/mil_dce.c,v retrieving revision 1.34 diff -u -r1.34 mil_dce.c --- pathfinder/compiler/mil/mil_dce.c 7 Jan 2010 15:24:28 -0000 1.34 +++ pathfinder/compiler/mil/mil_dce.c 14 Apr 2010 10:33:27 -0000 @@ -474,6 +474,7 @@ case m_error: case m_print: case m_printf: + case m_printerr: case m_break: case m_use: case m_update_tape: Index: pathfinder/compiler/mil/milprint.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint.c,v retrieving revision 1.101 diff -u -r1.101 milprint.c --- pathfinder/compiler/mil/milprint.c 7 Jan 2010 15:24:28 -0000 1.101 +++ pathfinder/compiler/mil/milprint.c 14 Apr 2010 10:33:27 -0000 @@ -28,6 +28,7 @@ | 'ERROR (' expression ')' <m_error> | 'print (' args ')' <m_print> | 'printf (' args ')' <m_printf> + | 'printerr (' args ')' <m_printerr> | 'col_name (' expr ',' expr ')' <m_col_name> | 'destroy_ws (' expression ')' <m_destroy_ws> | 'end_ws (' expression ',' expression ')' <m_end_ws> @@ -673,6 +674,13 @@ milprintf (")"); break; + /* statement: 'fprintf ( stderr (), ' expression ')' */ + case m_printerr: + milprintf ("fprintf ( stderr (),"); + print_args (n->child[0]); + milprintf (")"); + break; + case m_col_name: print_expression (n->child[0]); milprintf (".%s (", ID[n->kind]); Index: pathfinder/compiler/mil/milprint_summer.c =================================================================== RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milprint_summer.c,v retrieving revision 1.438 diff -u -r1.438 milprint_summer.c --- pathfinder/compiler/mil/milprint_summer.c 12 Jan 2010 09:22:14 -0000 1.438 +++ pathfinder/compiler/mil/milprint_summer.c 14 Apr 2010 10:33:27 -0000 @@ -11616,7 +11616,7 @@ " ERROR(err); -(try);\n"\ "} else if (genType.startsWith(\"timing\")) {\n"\ " time_print := usec() - time_print;\n"\ - " printf(\"\\nTrans %% 10.3f msec\\nShred %% 10.3f msec\\nQuery %% 10.3f msec\\n" LASTPHASE " %% 10.3f msec\\n\","\ + " fprintf(stderr(),\"\\nTrans %% 10.3f msec\\nShred %% 10.3f msec\\nQuery %% 10.3f msec\\n" LASTPHASE " %% 10.3f msec\\n\","\ " dbl(time_compile)/1000.0, dbl(time_shred)/1000.0, dbl(time_exec - time_shred)/1000.0, time_print/1000.0);\n}" const char* PFstopMIL(int statement_type) { Index: pathfinder/runtime/pathfinder.mx =================================================================== RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v retrieving revision 1.480 diff -u -r1.480 pathfinder.mx --- pathfinder/runtime/pathfinder.mx 12 Apr 2010 09:20:51 -0000 1.480 +++ pathfinder/runtime/pathfinder.mx 14 Apr 2010 10:33:27 -0000 @@ -6336,7 +6336,7 @@ /* execute query */ xquery = p+1; if (p[1] == '4' && p[2] == '2' && p[3] == '\n' && p[4] == 0 && genType == buf && output[2] == 0) { /* use '42' as latency query test */ - stream_printf(GDKout, "42\n\nTrans 0.000 msec\nShred 0.000 msec\nQuery 0.000 msec\nPrint 0.000 msec\n"); + stream_printf(GDKerr, "42\n\nTrans 0.000 msec\nShred 0.000 msec\nQuery 0.000 msec\nPrint 0.000 msec\n"); } else { err = xquery_change_genType(ctx, genType); if (err == NULL) ======== > > Index: MapiClient.mx > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/MapiClient.mx,v > retrieving revision 1.201 > retrieving revision 1.202 > diff -u -d -r1.201 -r1.202 > --- MapiClient.mx 29 Mar 2010 18:45:15 -0000 1.201 > +++ MapiClient.mx 13 Apr 2010 12:43:04 -0000 1.202 > @@ -332,10 +332,11 @@ > static void > timerEnd(void) > { > + stream_flush(toConsole); > t1 = gettime(); > if (mark && specials == NOmodifier) { > - stream_printf(toConsole, "%s %7ld.%03ld msec %s\n", mark, (long) ((t1 - t0) / 1000), (long) ((t1 - t0) % 1000), mark2 ? mark2 : ""); > - stream_flush(toConsole); > + fprintf(stderr, "%s %7ld.%03ld msec %s\n", mark, (long) ((t1 - t0) / 1000), (long) ((t1 - t0) % 1000), mark2 ? mark2 : ""); > + fflush(stderr); > } > } > > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 1

[Monetdb-developers] select data type to store variable length string
by kss ram 12 Apr '10

12 Apr '10

Hello, Thank you everyone for being supportive! I have a brief query about the data types, according to the documentation http://monetdb.cwi.nl/SQL/Documentation/Data-Types.html, for storing string/char type date, we have following options 1.CHAR[ACTER] (L) - character string with length L 2.VARCHAR (L) | CHARACTER VARYING (L) - string with atmost length L CLOB | CHARACTER LARGE OBJECT BLOB | BINARY LARGE OBJECT DECIMAL(P,S) | NUMERIC(P,S) My question is would CLOB would be the right choice to store variable length/ dynamic string/char data, varchar(L) is out of the picture as i wont know the length of the string. Thank you for your time and help!

2 1

[Monetdb-developers] regarding debug output in query execution
by anshul gangwar 12 Apr '10

12 Apr '10

hello Is there a cluster(column consists of many clusters) based joining. If so than is there different algo used for each cluster also. What is the difference between (b=tmp_2607) and b=tmpr_2610 Is there any big differece between them. And any time real BAT ids of (id given at time of creation) columns are passed to BAT_fetchjoin i.e algos or the bat id passed are only of the clusters. In which file we can find the code where mil code(generated physical plan from pf) code given as input and algo finding process starts Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php

2 1

[Monetdb-developers] Fw: regarding debug output in query execution
by anshul gangwar 12 Apr '10

12 Apr '10

Does anybody have any idea about our problem as asked before hello Is there a cluster(column consists of many clusters) based joining. If so than is there different algo used for each cluster also. What is the difference between (b=tmp_2607) and b=tmpr_2610 Is there any big differece between them. And any time real BAT ids of (id given at time of creation) columns are passed to BAT_fetchjoin i.e algos or the bat id passed are only of the clusters. In which file we can find the code where mil code(generated physical plan from pf) code given as input and algo finding process starts Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php -----Inline Attachment Follows----- ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev -----Inline Attachment Follows----- _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php

1 0

[Monetdb-developers] regarding monetdb data type
by Tomas Lopez 12 Apr '10

12 Apr '10

Hi, i installed MonetDB server and client. it works fine with console to create table and insert data. but if insert data larger than 512 byte, i can not get the whole data back to excel. can someone tell me what are the things that may be going wrong. thank u.

3 2

[Monetdb-developers] How to get BLOB data from ODBC?
by Xiao Lei 12 Apr '10

12 Apr '10

Hi guys, I tried to insert and get BLOB data from ODBC with Monet. But I always get the error message as, ERROR [07006] [MonetDB][ODBC Driver 1.0]Restricted data type attribute violation Can anyone tell me what's the problem? Thx. -- Best Regards, Xiao Lei

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_bat.mx, , 1.241, 1.242
by Niels Nes 11 Apr '10

11 Apr '10

On Sun, Apr 11, 2010 at 11:47:41AM +0200, Sjoerd Mullender wrote: > Should this have been changed on the Feb2010 branch? not sure, the problem of performance problems and leaking hash files was reported on the Head. I'll check the code in stable. Niels > > On 2010-04-11 11:30, Niels Nes wrote: > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv4560 > > > > Modified Files: > > gdk_bat.mx > > Log Message: > > make sure we use the correct hash file (use views bat id for hhash/thash) > > > > > > Index: gdk_bat.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bat.mx,v > > retrieving revision 1.241 > > retrieving revision 1.242 > > diff -u -d -r1.241 -r1.242 > > --- gdk_bat.mx 22 Feb 2010 10:10:48 -0000 1.241 > > +++ gdk_bat.mx 11 Apr 2010 09:30:13 -0000 1.242 > > @@ -3355,7 +3355,7 @@ > > > > hp->filename = GDKmalloc(strlen(nme) + 12); > > if (hp->filename != NULL) > > - sprintf(hp->filename, "%s.%chash", nme, b->batCacheid > 0 ? 'h' : 't'); > > + sprintf(hp->filename, "%s.%chash", nme, bv->batCacheid > 0 ? 'h' : 't'); > > if (hp->filename == NULL || > > (bv->H->hash = HASHnew(hp, tpe, yy + lim, HASHmask(lim))) == NULL) { > > GDKfree(hp->filename); > > > > > > ------------------------------------------------------------------------------ > > Download Intel® Parallel Studio Eval > > Try the new software tools for yourself. Speed compiling, find bugs > > proactively, and fine-tune applications for parallel performance. > > See why Intel Parallel Studio got high marks during beta. > > http://p.sf.net/sfu/intel-sw-dev > > _______________________________________________ > > Monetdb-checkins mailing list > > Monetdb-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > -- > Sjoerd Mullender > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

Re: [Monetdb-developers] Compiler Mapi.h on windows
by Ken Kao 11 Apr '10

11 Apr '10

Sorry, I confuse it . Mapi.h only includes stdio.h. ( Mapi.c does indeed include more headers.) My version is latest Feb2010-SP1 release of MonetDB. The really problem for me is that I can not Compiler my code on windows after include Mapi.c . There are many headers include in Mapi.c. (ex: monet_utils.h, stream.h, stream_socket.h) I Find them in < MonetDB-1.36.3\src\common > . I try to include them in my code as well. Unfortunately, the first problem is that I have to extended to include unistd.h in stream.h. I have no ideal to sovle it because unistd.h is a linux header. My compiler enviroment is visual studio 2008 . Could you give some guides to create a correct compiler enviroment for Mapi.c ? I does not understand that How can I create a similar build enviroment for my code as MonetDB on Windows .Could you give some suggestion ? I really hope to be able to compile Mapi.c on Windows myself. I also try to use static link to libmapi.dll to achieve my purpose . But I need libmapi.lib can be successfully. If I only have mapi.h and libmapi.dll . Can I successfully complete such a way ? If No. Can I have any way to get libmapi.lib ? I really need it to achieve the second method. Thank you very mch!!

1 0

[Monetdb-developers] help regarding installtion after changes
by anshul gangwar 10 Apr '10

10 Apr '10

hello If the installation fails , how to remove the changes that installation has done. We are doing the changes to file gdk_relop.c and its .mx one and then using supertar ball we are trying to install it. Is it also possible that it doesn't install fully and still say installed successfully as earlier we have installed mserver separately from synaptic manager in ubuntu and it says succesfullly installed Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php

3 2

[Monetdb-developers] Mapi query parameter constness
by Isidor Zeuner 09 Apr '10

09 Apr '10

Dear MonetDB developers, as I have been working on interfacing Mapi to C++ in a manner as lightweight as possible, I have noticed that the parameter binding interface does not interface to STL strings as seamlessly as it probably could. Similar considerations might well apply to other C-based string APIs: The STL string API provides accessor functions to the string's char data and its size. To ensure secure programming, the pointer to the char data has to be const. Yet, the Mapi interface for passing string parameters expects a writable char pointer. I'm not sure how future plans for Mapi look like, but currently thsi doesn't seem necessary to me. I have attached a patch to have Mapi treat the data as const, which works well for me. I also noticed that Mapi expects the string size parameter to be passed by reference. Is this necessary? Having an interface which supports pass-by-value would facilitate string API integration even more. Best regards, Isidor

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src Makefile.ag, , 1.17, 1.18
by Stefan Manegold 09 Apr '10

09 Apr '10

On Fri, Apr 09, 2010 at 10:28:06AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/clients/src > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv15735/src > > Modified Files: > Makefile.ag > Log Message: > don't try to compile ODBC if configure already concluded it won't fly, or if the user actually requested it not to be built Am I right that on system where configure is not available (read: Windows), we'll now have to set HAVE_UNIXODBC (provided we do have ODBC) "by hand" to have odbc compiled on such system(s)? Stefan > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/Makefile.ag,v > retrieving revision 1.17 > retrieving revision 1.18 > diff -u -d -r1.17 -r1.18 > --- Makefile.ag 7 Jan 2010 15:24:16 -0000 1.17 > +++ Makefile.ag 9 Apr 2010 10:28:04 -0000 1.18 > @@ -15,4 +15,4 @@ > # Copyright August 2008-2010 MonetDB B.V. > # All Rights Reserved. > > -SUBDIRS = mapilib mapiclient odbc HAVE_PERL?perl php HAVE_PYTHON?python HAVE_RUBYGEM?ruby examples HAVE_MAPILITE?mapilite > +SUBDIRS = mapilib mapiclient HAVE_UNIXODBC?odbc HAVE_PERL?perl php HAVE_PYTHON?python HAVE_RUBYGEM?ruby examples HAVE_MAPILITE?mapilite > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 1

[Monetdb-developers] other optimizer pipelines
by Wouter Alink 09 Apr '10

09 Apr '10

Dear M5 developers, Yesterday I reported a bug regarding the minimal optimizer pipe (ID: 2983773). And, as expected, I was asked to add a test for the issue, which I would like to do. The bug is triggered as follows: - start an mserver5 instance with the optimizer pipe set to the minimal pipe: "inline,remap,deadcode,multiplex" - connect with an mclient - connect again with an mclient This is obviously very easy to reproduce. But, I couldn't find any other test that sets a different optimizer upon startup of the mserver. I found a few tests that change the optimizer within a SQL script, but this will very likely not trigger bug (ID: 2983773). Then I was wondering; there are currently a few different optimizer pipelines defined in the monetdb5.conf. Isn't it a good idea to run at least a few (preferrably all) tests for each of those optimizer pipelines? I can imagine that testing all optimizer pipelines will take too much time for nightly testing, but running the testweb with different pipelines would probably trigger most obvious bugs that are currently found one-by-one. Bug #2983773 would most probably have been detected too. Anyway, I would like to add a test, and I guess testing multiple optimizer pipelines won't be high on the priority list. Therefore, could somebody perhaps point out how I could specify in the 'prologue' which optimizer pipeline to use? I did find a python file "sql/src/test/Connections/Tests/connections.py" which perhaps could be used? What is the preferred way for adding a test with a different optimizer path? Kind regards, Wouter

2 1

[Monetdb-developers] Compiler Mapi.h on windows
by Ken Kao 09 Apr '10

09 Apr '10

Deal all, I have to access data in MonetDB from C++ on windows , so I try to include Mapi.h and Mapi.c in my code . But Mapi.h must include many Linux header file (ex: unistd.h) . I can not Compiler my code after include Mapi.h and Mapi.c on windows. More seriously I can not access data in MonetDB from C++ on windows. If anyone had this experience . Please give me some suggestion to Solve the simple but big problem. Thank you very much!!

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, Feb2010, 1.478.2.4, 1.478.2.5
by Ying Zhang 09 Apr '10

09 Apr '10

On Fri, Apr 09, 2010 at 11:15:08AM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sfp-cvsdas-4.v30.ch3.sourceforge.com:/tmp/cvs-serv30978 > > Modified Files: > Tag: Feb2010 > pathfinder.mx > Log Message: > leftfetchjoin may return a view (in this case on idx_timestamps), > preventing the replace on idx_timestamps later on. > This fixes the problem that > doc("http://...") > gives a "no such document" error. No need to add test for this. It is already included by several tests, e.g., WebSite/Tests/books1.xq WebSite/Tests/HelloWorld9.xq WebSite/Tests/books2.xq WebSite/Tests/slashdot2.xq WebSite/Tests/books3.xq WebSite/Tests/slashdot.xq WebSite/Tests/HelloWorldA.xq The testweb results (of the candidate branch) show that HelloWorld9.xq, HelloWorldA.xq and books1.xq fail with this error, *only* on Fedora10 G.64.64.d.0 Jennie > > > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.478.2.4 > retrieving revision 1.478.2.5 > diff -u -d -r1.478.2.4 -r1.478.2.5 > --- pathfinder.mx 29 Mar 2010 10:38:24 -0000 1.478.2.4 > +++ pathfinder.mx 9 Apr 2010 11:15:06 -0000 1.478.2.5 > @@ -3284,7 +3284,7 @@ > > # for not-yet-cached URIs: get lifetime from uri_lifetime BAT > var idx_idx := idx_doc.ord_uselect(oid_nil).mirror(); > - var idx_ts := idx_idx.leftfetchjoin(idx_filenames).ord_uselect(str_nil).mirror().leftfetchjoin(idx_timestamps); > + var idx_ts := idx_idx.leftfetchjoin(idx_filenames).ord_uselect(str_nil).mirror().leftfetchjoin(idx_timestamps).copy(); > idx_ts@batloop() { # URI (filename = nil) > var b := [startsWith](idx_names.find($h), mirror(uri_lifetime)).uselect(true); > var lifetime := lng_nil; > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

1 0

[Monetdb-developers] maximum column name length
by kss ram 08 Apr '10

08 Apr '10

Hello, i am just starting up with monetdb, i have been going through the documentation and other resource but could not trace the information on this, what is the maximum size/length of the column name? Thank you. Saurabh

2 1

[Monetdb-developers] maximum column name length
by kss ram 08 Apr '10

08 Apr '10

Hello, i am just starting up with monetdb, i have been going through the documentation and other resource but could not trace the information on this, what is the maximum size/length of the column name? Thank you. Saurabh

1 0

[Monetdb-developers] Announcement: New Feb2010-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 08 Apr '10

08 Apr '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2010-SP1 bug fix release of the MonetDB suite of programs. More information on this release is available at <http://monetdb.cwi.nl/Development/Releases/Feb2010/>. -- Sjoerd Mullender

2 1

[Monetdb-developers] How can I get the libmapi.lib file
by KEN 08 Apr '10

08 Apr '10

Hi , I have to make my application programs (MFC C++) access data in MonetDB/XQuery on windows . I try to use static link by libMapi.dll ( the file in MonetDB4-XQuery-Installer ), but I can not find libMapi.lib in any package at Download area . Please give me a hand to get libMapi.lib or other ways. Let my MFC C++ access data easily in MonetDB/XQuery on windows. Thank you very much!!

2 1

[Monetdb-developers] dealing with 1G XML file in MonetDB/XQuery
by Mo Zhou 08 Apr '10

08 Apr '10

Hi, When I load a 1G XML file to MonetDB/XQuery, it throws exceptions as follows. I did not find related parameters in the monetDB config file to enlarge the heap size. I wonder how I can solve this problem? Thank you. #GDKmmap(1073741824) fails, try to free up space [memory in use=156511940,virtual memory in use=1309212672] #GDKmmap(1073741824) result [mem=156511940,vm=1309212672] MAPI = mozhou@localhost:40000 QUERY = pf:add-doc("/.../new1GL1Comp/xmldoc3.xml","1GL1doc3") ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 1073741824 for 03/02/30243theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text](http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor20/Offer31281/), BUNappend fails !ERROR: CMDshred_url: operation failed. -- Thanks, Mo

2 2

[Monetdb-developers] ~~Hi~~
by Erin's Ghost 08 Apr '10

08 Apr '10

http://sites.google.com/site/df3retegtr/klds3k

1 0

[Monetdb-developers] Access data in MonetDB/XQuery by MFC C++ on windows
by KEN 08 Apr '10

08 Apr '10

Hi , I have to make my application programs (MFC C++) access data in MonetDB/XQuery on windows . I try to use static link by libMapi.dll ( the file in MonetDB4-XQuery-Installer ), but I can not find libMapi.lib in any package at Download area . Please give me a hand to get libMapi.lib or other ways. Let my MFC C++ access data easily in MonetDB/XQuery on windows. PS : I find mapi.h and mapi.c , but the header file must includ some linux header (ex: unistd.h). I can not compiler it on windows. Thank you very much!!

1 0

[Monetdb-developers] ODBC with MonetDB
by Xiao Lei 08 Apr '10

08 Apr '10

Hi guys, I'm trying to insert data into a table of MonetDB from ODBC at windows platform. This is the definition of table, create table tbl1 (id int,dta clob); insert into tbl1 (id,dta) values(1,'[some thing]......') Here is the problem, If the length of [some thing] is bigger than 511, I can not get the entire result of "dta" from the table with sql sentence like "select * from tbl1". It would only return some part of the result(front part, like only 80 bytes. But it would work well if I insert data less than 511 bytes. Can anyone tell me how to deal with it? BTW, is there any way that I can use C# directly without ODBC? Thanks a lot. -- Best Regards, Xiao Lei

1 0

[Monetdb-developers] SQL leaking memory?
by Isidor Zeuner 07 Apr '10

07 Apr '10

Dear MonetDB developers, some time ago, I have noticed the 1.350 revision of sql/src/backends/monet5/sql_gencode.mx ([1]) introducing a leak warning. The recent 1.358 revision ([2]) also does memory-related changes. Does the warning still reflect the current status with the most recent CVS status? If yes, are there already plans on when SQL/MonetDB5 is supposed to be leak-free again? Best regards, Isidor Zeuner [1] http://monetdb.cvs.sourceforge.net/viewvc/monetdb/sql/src/backends/monet5/s… [2] http://monetdb.cvs.sourceforge.net/viewvc/monetdb/sql/src/backends/monet5/s…

1 0

[Monetdb-developers] regarding monetdb
by Puneet Birwa 06 Apr '10

06 Apr '10

hello, I am using ALGODEBUG THRprintf for printing & updating the MonetDB/MonetDB/src/gdk/gdk_relop.mx file but then also after installing monetdb I am unable to find any change in the output when I run monetdb in debug mode = 2097152. I have source code of monetDB & a script named as monetdb-install.sh I firstly run this script as ./monetdb-install.sh --enable-xquery. after its procedure gets completed i need to install few more things with the help of synaptic manager, that are as follows: 1) libmonetdb4-xquery0 2) monetdb4-server 3) monetdb5-sever 4) libmonetdb4-server4 5) libmonetdb5-sever5 6) libmonetdb-client1 7) libmonetdb1 8) monetdb-client 9) monetdb-xquery Previously i was trying to use ecllipse c/c++ version to run monetdb but i wasn't able to find the way of doing so. thats the reason I m doing the change in the source code & installing it again . But now also I cann't see the changes which I m making in source code . plz if anyone can tell me that why is it not working correctly & how can I make it work correctly. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

3 2

[Monetdb-developers] simple sql corrupts db
by Mark Bucciarelli 06 Apr '10

06 Apr '10

a couple weeks ago i posted two bug reports that show how a simple sql script can corrupt the database (that is, trigger load_idxcolumn: `kc->c' assertion failures when connecting). the reports include a test script that duplicated the behavior 100% of the time on my install (Feb2010). i'm reposting the scripts below as it seems there are more eyeballs here than on tracker. is one of the drop clauses i used unsupported? is there a workaround? thanks, m $ ./test.sh Creating Monet database tst created database in maintenance mode: tst starting database 'tst'... done taken database out of maintenance mode: tst stopping database 'tst'... done starting database 'tst'... FAILED start: starting 'tst' failed: database 'tst' has crashed after starting, manual intervention needed, check merovingian's logfile for details $ tail -3 /var/log/merovingian.log 2010-03-23 00:09:46 ERR tst[27143]: mserver5: store.mx:373: load_idxcolumn: Assertion `kc->c' failed. 2010-03-23 00:09:46 MSG merovingian[28274]: database 'tst' (27143) was killed by signal 6 2010-03-23 00:09:56 ERR control[28274]: (local): failed to fork mserver: database 'tst' has crashed after starting, manual intervention needed, check merovingian's logfile for details $ cat test.sh #! /bin/sh -e # # Create tst database in MonetDB. # user=monetdb pass=monetdb db=tst schema=tst # # Note: doesn't abort if database already exists. # echo "Creating Monet database ${db}" sudo monetdb create ${db} || exit 1 sudo monetdb start ${db} || exit 1 sudo monetdb release ${db} export DOTMONETDBFILE="./.testmonetdb" cat > ${DOTMONETDBFILE} << EOF user=${user} password=${pass} language=sql EOF sql="CREATE SCHEMA "${schema}" AUTHORIZATION "${user}";" mclient -d ${db} -s "${sql}" mclient -d ${db} < test.sql sudo monetdb stop tst sudo monetdb start tst #sudo monetdb stop tst && sudo monetdb destroy tst $ cat test.sql -- -- Setup two test tables, one that references the other. -- CREATE TABLE "tst"."t1" ( "id" int NOT NULL, CONSTRAINT "t1_id_pkey" PRIMARY KEY ("id") ); CREATE TABLE "tst"."t2" ( "id" int NOT NULL, "t1_id" int NOT NULL, CONSTRAINT "t2_id_pkey" PRIMARY KEY ("id"), CONSTRAINT "t2_t1_id_fkey" FOREIGN KEY ("t1_id") REFERENCES "tst"." t1" ("id") ); CREATE INDEX "t2_t1_id" ON "tst"."t2" ("t1_id"); -- -- Drop fkey constraint and index -- DROP INDEX "tst"."t2_t1_id"; ALTER TABLE "tst"."t2" DROP CONSTRAINT t2_t1_id_fkey; -- -- Drop fkey column that links tables. -- ALTER TABLE "tst"."t2" DROP COLUMN t1_id; $

1 0

[Monetdb-developers] regarding monetdb
by Puneet Birwa 06 Apr '10

06 Apr '10

hello, i have the code of monetdb & I m doing some changes in MonetDB/MonetDB/src/gdk/gdk_relop.c file & I m again installing the modified monetdb. But when I am running this monetdb in debug mode = 2097152 it is not showing the changes in the output it generates. The changes are simple & I just want to print few things but I m unable to do that . can someone tell me what are the things that may be going wrong. thank u. Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] running after modification and debugging monetdb
by Puneet Birwa 05 Apr '10

05 Apr '10

hello I want to know how to run the monetdb from scratch after doing the modifications to source files and debugging the course of execution of it The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] regarding execution plan
by Puneet Birwa 04 Apr '10

04 Apr '10

hello, I want to make graph or tree of the final plan which monetdb uses to execute the xquery. I am running the monetdb server in debug mode = "2097152" & I am trying to make graph/tree of the output which I am getting in this mode but unable to do. So can any one help me in finding the graph/tree of the final plan which monetdb uses to execute the query . plz reply as soon as possible. thank you. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] regarding debugging
by Puneet Birwa 04 Apr '10

04 Apr '10

hello, can i use ecllipse for debugging the monetdb code. i.e can i run the monetdb with the help of ecllipse. if yes please can somebody tell how can i do it. plz reply soon . thank u. Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 2

[Monetdb-developers] the database and index size
by Mo Zhou 30 Mar '10

30 Mar '10

Hi, When I create a database and load the data from plain files, does monetdb create indexes automatically? When I examine the disk usage of a database in dbfarm, does it include the indexes size, metadata, and all tables' sizes? -- Thanks, Mo

2 1

[Monetdb-developers] regarding monetdb
by Puneet Birwa 30 Mar '10

30 Mar '10

hi, is monetdb mantain the hash table & indexs for each node by itself. & is there any way by which i can find out the relation between the physical plan generated at compile time & execution plan which is generated at run time i.e how can i find mapping between physical plan & execution plan . Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

1 0

[Monetdb-developers] sourceforge outage
by Sjoerd Mullender 26 Mar '10

26 Mar '10

See the current status at https://sourceforge.net/apps/wordpress/sourceforge/. -- Sjoerd Mullender

1 0

[Monetdb-developers] dlclose() segfault
by Mark Bucciarelli 23 Mar '10

23 Mar '10

While learning about dlopen() and dynamic libraries, I found that calling dlclose() on libbat.so segfaults. Is this a known behavior or a symptom of some problem? Thanks, m $ cat bad.c #include <dlfcn.h> int main(void) { void *handle; char *fn = "libbat.so"; handle = dlopen(fn, RTLD_NOW); dlclose(handle); return 0; } $ gcc bad.c && ./a.out Segmentation fault (core dumped) $

2 3

[Monetdb-developers] C client: MapiHdl usage
by Mark Bucciarelli 23 Mar '10

23 Mar '10

Hi, Is there any reason I cannot reuse MapiHdl? Do I need to close after an insert or a mapi_get_last_id() call? Thanks, m

2 1

[Monetdb-developers] are BAT ids are fixed
by anshul gangwar 22 Mar '10

22 Mar '10

hello, I want to know whether BAT ids are fixed. If fixed then what does that BAT_select_(b=tmpr_2322): BATslice(v=tmp_2403, low=0, high=1) means? The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] difficulty in understanding final query execution plan
by anshul gangwar 22 Mar '10

22 Mar '10

Hello What will be the leaves in final query execution plan tree. What does that BATfetchjoin does , BAT_select and BATjoin actually does in simple language. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] BAT_select description
by anshul gangwar 22 Mar '10

22 Mar '10

hello Where can i find the description of BAT_select? Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/kernel batcast.mx, Feb2010, 1.31.2.7, 1.31.2.8
by Stefan Manegold 20 Mar '10

20 Mar '10

On Sat, Mar 20, 2010 at 12:53:12PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv10537 > > Modified Files: > Tag: Feb2010 > batcast.mx > Log Message: > Backport of possible dangerous double frees. No roll forward needed. While your revised code looks indeed more efficient to me (as it allows ATOMfromstr() --- well, at least strFromStr(); I cannot tell about other "extern" (varsized) atoms --- to reuse a allocated temporary buffer), I wonder whether/why the original experienced double frees? Called with v==NULL, ATOMfromstr() --- again, I refer only to strFromStr(); FromStr functions for other "extern" (varsized) atoms might behave differently; IMHO, we don't have any "contract" or "policies" for FromStr() implementations --- would allocate a buffer for the result, ATOMput() copies the value from the buffer to the BAT heap, and then the buffer is (was) freed. Looks very sound to me. What am I missing? I would be thankful for any enlightment. Stefan > Index: batcast.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/batcast.mx,v > retrieving revision 1.31.2.7 > retrieving revision 1.31.2.8 > diff -u -d -r1.31.2.7 -r1.31.2.8 > --- batcast.mx 18 Feb 2010 01:04:05 -0000 1.31.2.7 > +++ batcast.mx 20 Mar 2010 12:53:10 -0000 1.31.2.8 > @@ -248,6 +248,8 @@ > BAT *b,*bn; > BATiter bi; > BUN p, q, r = 0; > + ptr v = NULL; > + int len = 0; > > @:getBATdescriptor(bid, b, "batcalc.@1")@ > @:voidresultBAT(TYPE_@1,"batcalc.@1")@ > @@ -255,15 +257,12 @@ > > BATaccessBegin(b, USE_TAIL, MMAP_SEQUENTIAL); > BATloop(b, p, q) { > - ptr v = NULL; > - int len = 0; > - > ATOMfromstr(TYPE_@1, &v, &len, (char *) BUNtvar(bi, p)); > ATOMput(TYPE_@1, bn->T->vheap, Tloc(bn, r), v); > - if (ATOMextern(TYPE_@1)) > - GDKfree(v); > r++; > } > + if (ATOMextern(TYPE_@1)) > + GDKfree(v); > BATsetcount(bn, BATcount(b)); > bunins_failed: > BATaccessEnd(b, USE_TAIL, MMAP_SEQUENTIAL); > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 3

Re: [Monetdb-developers] why open() before dlopen()?
by Mark Bucciarelli 19 Mar '10

19 Mar '10

On Fri, Mar 19, 2010 at 11:38 AM, Martin Kersten <Martin.Kersten(a)cwi.nl> wrote: > > Also portability over all platforms is point of concern, > there is some old old out there, e.g. HP-UX. > Ok, then I'll make the patch specific to OpenBSD unless someone here has a preferred solution. Thanks, m

2 2

[Monetdb-developers] why open() before dlopen()?
by Mark Bucciarelli 19 Mar '10

19 Mar '10

I'm working on fixing tracker #2971516. What I have learned is that libtool on OpenBSD does not create .so files without the version suffix because they use a different value of library_names_spec in libtool than Linux does. However, their dlopen() has some ld magic; for example, dlopen("libbat.so") and dlopen("/usr/local/lib/libbat.so.1.34") both work just fine. However the monetdb server tries to open() the .so first, and the open() method does not have the same ld magic as dlopen(). So my question is---why call open() at all? What value does it add to the dlopen() call? Seems like picking the right library version is a responsibility that should belong to ld. I'm contemplating a patch that skips the open() call for .so files. Thanks, m

1 0

[Monetdb-developers] IMPORTANT: changes are afoot
by Sjoerd Mullender 18 Mar '10

18 Mar '10

For a long time now we have had serious performance issues with the MonetDB CVS repository on Sourceforge. MonetDB B.V. has bought a machine that is to be used as the host for the public repository for MonetDB. This means that at some as yet undecided point in the near future (i.e. fairly soon) we will move the repository away from Sourceforge and to this new machine. In the process we will also convert the current CVS repository to a new and more modern type, namely Mercurial (also known as hg). Mercurial is a so called distributed version control system (DVCS). This means that every developer has his/her own complete copy of the repository and can make any changes they want to it. In order to share any results, developers with the right access rights can "push" their changes to the central repository, and anyone can "pull" changes from that repository to their own. Mercurial (like its competitors git and bazaar) is very good at managing this push/pull work, merging changes from different sources and keeping track of what has actually changed (as opposed to what the end result looks like, which is what CVS and Subversion (SVN) do). Read-only access to the Mercurial repository will of course be possible for anyone. This only requires the HTTP protocol. Read-write access to the repository will require an SSH public/private key pair (of which you will need to keep the private part, well, private; the public part will need to be sent to me--I will contact you for that). Because Mercurial is distributed, we can work in a slightly different way than we used to with CVS. A group of core developers will have read-write access. They can make their copy of the repository (or one of their copies) available for their students who can develop and push their changes to this copy. Once the core developer is satisfied with the changes he/she can push those changes to the main repository. This way of working has several big advantages: - Any developer can store changes (with appropriate log messages) in their copy of the repository, safeguarding those changes, even if they are not yet fully developed. - Once changes are mature enough to share, they can be shared. - We don't need to give lots and lots of people write access to the main repository. Currently there are a large number of people with write access to the repository at Sourceforge of whom I don't know whether they still need the access. The new machine will do more than just serving the repository. It will also be used as the MonetDB webserver, and we have plans to switch over to another bug tracking system (bugzilla) that is to run on this system. An interesting intro into Mercurial usage can be found here: <http://hginit.com/00.html>. -- Sjoerd Mullender

1 0

[Monetdb-developers] locate_file and error handling
by Mark Bucciarelli 17 Mar '10

17 Mar '10

Hi, As I read the source, I see lots of instances where return values are not checked or not handled; for example, strdup(). Most recently, this programming idiom triggered a null pointer dereference, which caused mserver to segfault (see tracker #2971516 [1]). In my experience, when the logic encounters a condition it can't recover from, exiting quickly with as much information as possible is very useful. Whe I read locate_file() method and see: char *mod_path = GDKgetenv("monet_mod_path"); if (mod_path == NULL) return NULL; I wonder---how can you recover from this? I want to submit a patch that says: if (mod_path == NULL) errx(1, "locate_file: GDBgetenv("monet_mod_path") is null"); But I'm not sure you guys will accept this. Would you? It's a large change to error handling style. What happens to me in #2971516 is simply the library file name is not found. But instead of a useful error message that tells me what directories where searched and what the file name is, I get ERR control[14875]: (local): failed to fork mserver: database 'test' has inconsistent state (running but dead), review merovingian's logfile for any peculiarities and an mserver5.core file. Thanks, m [1] https://sourceforge.net/tracker/index.php?func=detail&aid=2971516&group_id=…

1 0

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 17 Mar '10

17 Mar '10

Yes we are on monetdb/xquery The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

3 2

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 15 Mar '10

15 Mar '10

hello , actually we are working on java based tool whose main focus is to visualize the plans used by the optimizer for final execution of query but in monetdb optimization is runtime so we have to somehow access that plan through jdbc and what are the maincomponents of the plan i.e can be represented as node in tree We are not able to use that , actually that is giving error MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !xchange_challenge: frontend mal not found Dear Anshul, Op 15-03-10 05:42, anshul gangwar schreef: > We want to debug the monetdb server , in other words we want final > execution plan and query execution times etc. Then most likely you will love the stethoscope tool! It gives an awesome view in the MonetDB kernel *and* it provides you realtime information about the queries that have been executed. That is not all, you can just start and stop it without(!) restarting the server, while still making nifty graphs from this program. I think this is totally what you need. There is a lot documentation written for this tool, so you don't have to be a rocket scientist to use it :) <http://monetdb.cwi.nl/MonetDB/Documentation/Stethoscope.html> The last line on the page will give you an example commandline. If you did not yet change your default monetdb password you can use this one: stethoscope -umonetdb -Pmonetdb +tis algebra.* bat.* group.* sql.* aggr.* Please report back how useful this was for your debuging and use of MonetDB. We appreciate your interest. Yours Sincerely, Stefan de Konink -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkudwdMACgkQYH1+F2Rqwn0+AwCfTg5FruAc6QDgOIT4ZJZ4bOWB hTAAnjkWylxX7JafSLI21qZFE6BMydoO =8rz4 -----END PGP SIGNATURE----- Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 15 Mar '10

15 Mar '10

We want to debug the monetdb server , in other words we want final execution plan and query execution times etc. > We are working on linux so any way through jdbc Just as a sidenote: JDBC vs OBDC is not necessarily a question of the OS; both JDBC and ODBC are available and supported on various OSs, among which Windows and Linux (there is UnixODBC for Linux). > Can we get the debugger output through jdbc Which debugger and/or debugger output are you referring to / talking about? Or in other words, what do you want to debug? Your JDBC client? The JDBC driver? The MonetDB server? The MonetDB kernel? Stefan > You can enable the ODBC debug output by your ODBC configuration. > > For example: > > Trace = Yes > TraceFile = /tmp/debug.log > ForceTrace = Yes > > > Stefan > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEAREKAAYFAkudHEYACgkQYH1+F2Rqwn2n1ACeJkxR8di3h1McltuRXsDZRxri > V9UAoIchyOqm2pqGDrCWITtWUJngAkhg > =Qz4/ > -----END PGP SIGNATURE----- > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/ > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 | The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] related to jdbc
by anshul gangwar 14 Mar '10

14 Mar '10

hello, Can we get the debugger output through jdbc Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 2

[Monetdb-developers] Fw: Re: related to jdbc
by anshul gangwar 14 Mar '10

14 Mar '10

We are working on linux so any way through jdbc Can we get the debugger output through jdbc You can enable the ODBC debug output by your ODBC configuration. For example: Trace = Yes TraceFile = /tmp/debug.log ForceTrace = Yes Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkudHEYACgkQYH1+F2Rqwn2n1ACeJkxR8di3h1McltuRXsDZRxri V9UAoIchyOqm2pqGDrCWITtWUJngAkhg =Qz4/ -----END PGP SIGNATURE----- ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/conf monetdb5.conf.in, , 1.86, 1.87
by Martin Kersten 14 Mar '10

14 Mar '10

Fabian Groffen wrote: > On 14-03-2010 08:23:57 +0000, Martin Kersten wrote: >> Update of /cvsroot/monetdb/MonetDB5/conf >> In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv1052 >> >> Modified Files: >> monetdb5.conf.in >> Log Message: >> The garbage collector optimizer should be the last in the pipelines. > > Was it really the intention to revert my commit making a separate no > minimal_pipe? If so, then also remove the changelog entry for it, I'd > say. > Also, why is there no garbage collector in the minimal pipe? it should > >> Index: monetdb5.conf.in >> =================================================================== >> RCS file: /cvsroot/monetdb/MonetDB5/conf/monetdb5.conf.in,v >> retrieving revision 1.86 >> retrieving revision 1.87 >> diff -u -d -r1.86 -r1.87 >> --- monetdb5.conf.in 9 Mar 2010 12:02:32 -0000 1.86 >> +++ monetdb5.conf.in 14 Mar 2010 08:23:53 -0000 1.87 >> @@ -255,47 +255,50 @@ >> # optimizers to run, or one or more variables containing the optimizer >> # pipeline to run. The latter is provided for readability purposes >> # only. >> +# >> +# The minimal pipeline necessary by the server to operate correctly: >> +# sql_optimizer=inline,remap,deadcode,multiplex >> >> # Set and define the default pipeline >> sql_optimizer=default_pipe >> >> -# The minimal pipeline necessary by the server to operate correctly >> -minimal_pipe=inline,remap,deadcode,multiplex >> - >> # The default pipe line contains as of Feb2010 mitosis-mergetable-reorder, aimed at large tables and improved access locality >> -default_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +default_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # the default pipeline used in the November 2009 release >> -nov2009_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +nov2009_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # experimental pipelines stressing various components >> >> # Replication >> -replication_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,replication,multiplex >> +replication_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector >> >> # Accumulator >> -accumulator_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,garbageCollector,dataflow,history,multiplex >> +accumulator_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,dataflow,history,multiplex,garbageCollector >> >> # The Recycle pipeline for Skyserver >> -recycler_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,garbageCollector,dataflow,history,multiplex >> +recycler_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,dataflow,history,multiplex,garbageCollector >> >> # The Cracking pipeline(s) >> -cracker_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> -sidcrack_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,sidcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +cracker_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> +sidcrack_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,sidcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # The datacell pipeline >> -datacell_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +datacell_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # The Octopus pipeline for distributed processing (Merovingian enabled >> # platforms only) >> -octopus_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +octopus_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mitosis,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> >> # >> #The default + datacyclotron >> -datacyclotron_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,datacyclotron,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,garbageCollector,dataflow,history,replication,multiplex >> +datacyclotron_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,datacyclotron,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,dataflow,history,replication,multiplex,garbageCollector >> # >> # the default + derivePath >> -derive_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,derivePath,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +derive_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,derivePath,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> # >> #the default + dictionary >> -dictionary_pipe=inline,remap,dictionary,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex >> +dictionary_pipe=inline,remap,dictionary,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,history,multiplex,garbageCollector >> +# >> +#the default + compression >> +compression_pipe=inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector >> >> >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> Monetdb-checkins mailing list >> Monetdb-checkins(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk.mx, , 1.310, 1.311 gdk_atoms.mx, , 1.176, 1.177 gdk_bbp.mx, , 1.275, 1.276
by Niels Nes 13 Mar '10

13 Mar '10

On Sat, Mar 13, 2010 at 11:05:37PM +0100, Sjoerd Mullender wrote: > How backward and forward compatible is this? You should probably also > change the GDKLIBRARY value and take care of reading older versions. should be backward compatible, ie if its not there, we have the old situation (at least that was the intention) > > On 2010-03-13 22:48, Niels Nes wrote: > > Update of /cvsroot/monetdb/MonetDB/src/gdk > > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19213/src/gdk > > > > Modified Files: > > gdk.mx gdk_atoms.mx gdk_bbp.mx > > Log Message: > > LLONG_MAX should be defined before compiling BUNlast (needs this because of > > BUN_NONE) > > > > (keep extra string with options in the BBP.dir, for dynamic storage options) > > > > > > Index: gdk_bbp.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v > > retrieving revision 1.275 > > retrieving revision 1.276 > > diff -u -d -r1.275 -r1.276 > > --- gdk_bbp.mx 7 Mar 2010 19:00:40 -0000 1.275 > > +++ gdk_bbp.mx 13 Mar 2010 21:48:43 -0000 1.276 > > @@ -701,7 +701,7 @@ > > int lastused; > > int nread; > > BATstore *bs; > > - char *s; > > + char *s, *options = NULL; > > char logical[1024]; > > lng inserted, deleted, first, count, capacity; > > unsigned short map_head, map_tail, map_hheap, map_theap; > > @@ -848,8 +848,10 @@ > > @:vheapinit(H)@ > > @:vheapinit(T)@ > > > > - if (buf[nread] != '\n') > > + if (buf[nread] != '\n' && buf[nread] != ' ') > > GDKfatal("BBPinit: invalid format for BBP.dir\n%s", buf); > > + if (buf[nread] == ' ') > > + options = buf+nread+1; > > > > BBP[bid].cache = bs; > > BBP[bid].status = BBPEXISTING; /* do we need other status bits? */ > > @@ -867,6 +869,9 @@ > > else > > BBP[bid].nme[1] = GDKstrdup(BBPtmpname(tailname, sizeof(tailname), -bid)); > > BBP[bid].path = GDKstrdup(filename); > > + BBP[bid].options = NULL; > > + if (options) > > + BBP[bid].options = GDKstrdup(options); > > BBP[bid].lastused = lastused; > > if (lastused > max_stamp) > > max_stamp = lastused; > > @@ -1052,6 +1057,9 @@ > > @:vheapprint(H)@ > > @:vheapprint(T)@ > > > > + if (BBP[i].options) > > + stream_printf(s, " %s", BBP[i].options); > > + > > return stream_printf(s, "\n"); > > } > > > > > > Index: gdk.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk.mx,v > > retrieving revision 1.310 > > retrieving revision 1.311 > > diff -u -d -r1.310 -r1.311 > > --- gdk.mx 10 Mar 2010 22:17:11 -0000 1.310 > > +++ gdk.mx 13 Mar 2010 21:48:38 -0000 1.311 > > @@ -389,6 +389,17 @@ > > # define WIFEXITED(stat_val) (((stat_val) & 255) == 0) > > #endif > > > > +#include <limits.h> /* for *_MIN and *_MAX */ > > +#include <float.h> /* for FLT_MAX and DBL_MAX */ > > +#ifndef LLONG_MAX > > +#ifdef LONGLONG_MAX > > +#define LLONG_MAX LONGLONG_MAX > > +#define LLONG_MIN LONGLONG_MIN > > +#else > > +#define LLONG_MAX LL_CONSTANT(9223372036854775807) > > +#define LLONG_MIN (-LL_CONSTANT(9223372036854775807) - LL_CONSTANT(1)) > > +#endif > > +#endif > > > > #ifdef __cplusplus > > extern "C" { > > @@ -2037,6 +2048,7 @@ > > bat next[2]; /* next BBP slot in link list */ > > BATstore *cache; /* cached header info */ > > str path; /* dir + basename for storage */ > > + str options; /* A string list of options */ > > int refs; /* in-memory references on which the loaded status of a BAT relies */ > > int lrefs; /* logical references on which the existence of a BAT relies */ > > int lastused; /* BBP LRU stamp */ > > @@ -2054,6 +2066,7 @@ > > #define BBP_logical(i) BBP[ABS(i)].nme[(i)<0] > > #define BBP_next(i) BBP[ABS(i)].next[(i)<0] > > #define BBP_physical(i) BBP[ABS(i)].path > > +#define BBP_options(i) BBP[ABS(i)].options > > #define BBP_desc(i) BBP[ABS(i)].cache > > #define BBP_refs(i) BBP[ABS(i)].refs > > #define BBP_lrefs(i) BBP[ABS(i)].lrefs > > > > Index: gdk_atoms.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_atoms.mx,v > > retrieving revision 1.176 > > retrieving revision 1.177 > > diff -u -d -r1.176 -r1.177 > > --- gdk_atoms.mx 12 Feb 2010 08:58:35 -0000 1.176 > > +++ gdk_atoms.mx 13 Mar 2010 21:48:42 -0000 1.177 > > @@ -718,17 +718,6 @@ > > The routine atomnil returns a pointer to the nil value representation. > > @{ > > @h > > -#include <limits.h> /* for *_MIN and *_MAX */ > > -#include <float.h> /* for FLT_MAX and DBL_MAX */ > > -#ifndef LLONG_MAX > > -#ifdef LONGLONG_MAX > > -#define LLONG_MAX LONGLONG_MAX > > -#define LLONG_MIN LONGLONG_MIN > > -#else > > -#define LLONG_MAX LL_CONSTANT(9223372036854775807) > > -#define LLONG_MIN (-LL_CONSTANT(9223372036854775807) - LL_CONSTANT(1)) > > -#endif > > -#endif > > #define GDK_bit_max ((bit) 1) > > #define GDK_bit_min ((bit) 0) > > #define GDK_chr_max ((chr) SCHAR_MAX) > > > > > > ------------------------------------------------------------------------------ > > Download Intel® Parallel Studio Eval > > Try the new software tools for yourself. Speed compiling, find bugs > > proactively, and fine-tune applications for parallel performance. > > See why Intel Parallel Studio got high marks during beta. > > http://p.sf.net/sfu/intel-sw-dev > > _______________________________________________ > > Monetdb-checkins mailing list > > Monetdb-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > -- > Sjoerd Mullender > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

1 0

[Monetdb-developers] how to find cost
by Puneet Birwa 11 Mar '10

11 Mar '10

hello, thanks for help, but also can u tell me any way to find the cost of final execution plan, m unable to find any option to see what is the actual cost of final execution plan. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] final execution plan
by Puneet Birwa 11 Mar '10

11 Mar '10

hello, can anyone tell me how in monetdb/xquery we can see what is the final plan with the help of which query is executed i.e how can i find the plan generated at run time for query execution (which tells that which join operation is performed at particular node). The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] A SERIOUS CONCURRENCY CONTROL BUG
by oursql 09 Mar '10

09 Mar '10

There is a bug in monetdb, you can run the following script in two clients(mclient.exe), and before test, please disable autocommit option. user A: USER B: CREATE TABLE TEST(C1 INT PRIMARY KEY, C2 INT); INSERT INTO TEST VALUES(1,1); COMMIT; SELECT C2 FROM TEST WHERE C1=1; SELECT C2 FROM TEST WHERE C1=1; UPDATE TEST SET C2=10 WHERE C1=1; UPDATE TEST SET C2=30 WHERE C1=1; COMMIT; COMMIT;--CONFLICT, NEED TO ROLLBACK; ROLLBACK; UPDATE TEST SET C2=30 WHERE C1=1; SELECT C2 FROM TEST WHERE C1=1; --THE RESULT IS 10, NOT 30. TOO BAD! 2010-03-09 oursql

1 0

[Monetdb-developers] monetdb cost model
by anshul gangwar 08 Mar '10

08 Mar '10

hello , Is the cost model used at compile time or at runtime for estimation of cardinalities Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] Fw: monetdb cost model
by anshul gangwar 08 Mar '10

08 Mar '10

hello , Is the cost model used at compile time or at runtime for estimation of cardinalities please reply as soon as soon as possible Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 2

[Monetdb-developers] need help in monetdb
by anshul gangwar 08 Mar '10

08 Mar '10

hello how can we get the final executed plan i.e at run time in monetdb. as pysical-algebra-plan only shows the result after compilation time. And at runtime what does it acually does i.e after sampling what does it decide i.e type of join or others thing also. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer/Tests ifthencst.stable.out, , 1.3, 1.4
by Stefan Manegold 07 Mar '10

07 Mar '10

FYI: this test has been failing in the Feb2010 branch since Feb 25 2010, i.e., possibly triggered by checkins on Feb 24 2010. http://monetdb.cwi.nl/testing/projects/monetdb/Stable/MonetDB5/.mTests103/G… Note that the output it (now) produces with the Feb2010 branch is different from what been approved here in the CVS head. Stefan On Sun, Mar 07, 2010 at 08:05:00PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer/Tests > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv3208/Tests > > Modified Files: > ifthencst.stable.out > Log Message: > Approve output > > > Index: ifthencst.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Tests/ifthencst.stable.out,v > retrieving revision 1.3 > retrieving revision 1.4 > diff -u -d -r1.3 -r1.4 > --- ifthencst.stable.out 17 Oct 2009 21:22:35 -0000 1.3 > +++ ifthencst.stable.out 7 Mar 2010 20:04:57 -0000 1.4 > @@ -26,8 +26,6 @@ > io.print("never executed"); > exit d; > end tst; > -[ "always executed" ] > -[ "always executed" ] > function user.main():void; > # some frontend generates the following code > mdb.list("user","tst"); > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] is there exist any histogram or any other statistic in monetdb/xquery
by Puneet Birwa 07 Mar '10

07 Mar '10

hello, can someone tell me that how monetdb keep track of cardinality of result of query for generating optimal plans or it is completely independent of histogram or anything which keep track of cardinality of result of query. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: need help in monetdb xquery interface
by anshul gangwar 07 Mar '10

07 Mar '10

hello, I just want to know about current monetdb version. Whether it does any optimization at compile time or not or it fully does at run time only. where can i get the information about the optimization strategy of monetdb. May be the specific src code file to refer of monetdb. can you please refer some research papers on monetdb optimization strategy etc. On Sat, Mar 06, 2010 at 07:27:09PM +0530, anshul gangwar wrote: > > > how can we get the estimate of the size of result of query based on any two attributes with varying rangeAnshul Well, I guess you'd need to browse the related research literature for XML/XQuery cost modeling / result size estimation techniques --- however, to my knowledge that area is not yet as well charted and understood as cost modeling / result size estimation for relational algebra and SQL. Of course, the details highly depend on your XML document structure and data distribution as well as the shape and characteristics of your XQuery queries ... Stefan > On Sat, Mar 06, 2010 at 11:32:03AM +0530, anshul gangwar wrote: > > > > > > --- On Fri, 5/3/10, Martin Kersten <Martin.Kersten(a)cwi.nl> wrote: > > > > From: Martin Kersten <Martin.Kersten(a)cwi.nl> > > Subject: * Re: [Monetdb-developers] need help in monetdb xquery interface > > To: "anshul gangwar" <anshul1886(a)yahoo.co.in> > > Date: Friday, 5 March, 2010, 11:26 PM > > > > anshul gangwar wrote: > > > hello , > > > I want to know whether there is any expression or command or anything equivalent to explain sql construct. Also What is the way to > > There is material for you to study on the website: > > http://monetdb.cwi.nl/SQL/Documentation/SQL-Runtime-Features.html > > http://monetdb.cwi.nl/SQL/Documentation/EXPLAIN-Statement.html#EXPLAIN-Stat… > > > get histogram statistics i.e way to find that histogram exists on attribute or not through Xquery interface. not through sql . > > MonetDB does not rely on separate statistics tables. > > > > > Please reply as soon as possible. > > > > > > thank you. > > > > > > Actually i want the xquery construct equivalent to sql explain construct and > > MonetDB/XQuery does not provide such feature. > > To see the equivalent of the SQL EXPLAIN output, i.e., the MIL plan > generated from your XQuery query, you need to use the Pathfinder stand-alone > compiler "pf"; see > `pf --help` > and/or > `pf --fullhelp` > for details. > > > so how it simulates histogram features i.e to get that percentage of query etc. > > Like MonetDB/SQL also > MonetDB/XQuery does not rely on separate statistics > such as histograms. > > Stefan > > > > > > Your Mail works best with the New Yahoo Optimized IE8. Get it NOW!. > > > Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/ > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 | The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

4 4

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: Fw: Re: need help in monetdb xquery interface
by anshul gangwar 07 Mar '10

07 Mar '10

hello, what are these statistical guides based on data guides in monetdb. And did monetdb stores these statistical guides(data guides). And if yes where it stores it & how can we access it . & does the physical algebra plan depend only on these statistical guides at compile time without actually running the query. And what are children Histogram & Typed valued Histogram , & are these Histograms are stored & if yes how can we access it. Your Mail works best with the New Yahoo Optimized IE8. Get it NOW!. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

1 0

[Monetdb-developers] Fw: Re: Fw: Re: need help in monetdb xquery interface
by anshul gangwar 06 Mar '10

06 Mar '10

how can we get the estimate of the size of result of query based on any two attributes with varying rangeAnshul On Sat, Mar 06, 2010 at 11:32:03AM +0530, anshul gangwar wrote: > > > --- On Fri, 5/3/10, Martin Kersten <Martin.Kersten(a)cwi.nl> wrote: > > From: Martin Kersten <Martin.Kersten(a)cwi.nl> > Subject: * Re: [Monetdb-developers] need help in monetdb xquery interface > To: "anshul gangwar" <anshul1886(a)yahoo.co.in> > Date: Friday, 5 March, 2010, 11:26 PM > > anshul gangwar wrote: > > hello , > > I want to know whether there is any expression or command or anything equivalent to explain sql construct. Also What is the way to > There is material for you to study on the website: > http://monetdb.cwi.nl/SQL/Documentation/SQL-Runtime-Features.html > http://monetdb.cwi.nl/SQL/Documentation/EXPLAIN-Statement.html#EXPLAIN-Stat… > > get histogram statistics i.e way to find that histogram exists on attribute or not through Xquery interface. not through sql . > MonetDB does not rely on separate statistics tables. > > > Please reply as soon as possible. > > > > thank you. > > > Actually i want the xquery construct equivalent to sql explain construct and MonetDB/XQuery does not provide such feature. To see the equivalent of the SQL EXPLAIN output, i.e., the MIL plan generated from your XQuery query, you need to use the Pathfinder stand-alone compiler "pf"; see `pf --help` and/or `pf --fullhelp` for details. > so how it simulates histogram features i.e to get that percentage of query etc. Like MonetDB/SQL also MonetDB/XQuery does not rely on separate statistics such as histograms. Stefan Your Mail works best with the New Yahoo Optimized IE8. Get it NOW!. Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] Fw: Re: need help in monetdb xquery interface
by anshul gangwar 06 Mar '10

06 Mar '10

--- On Fri, 5/3/10, Martin Kersten <Martin.Kersten(a)cwi.nl> wrote: From: Martin Kersten <Martin.Kersten(a)cwi.nl> Subject: Re: [Monetdb-developers] need help in monetdb xquery interface To: "anshul gangwar" <anshul1886(a)yahoo.co.in> Date: Friday, 5 March, 2010, 11:26 PM anshul gangwar wrote: > hello , > I want to know whether there is any expression or command or anything equivalent to explain sql construct. Also What is the way to There is material for you to study on the website: http://monetdb.cwi.nl/SQL/Documentation/SQL-Runtime-Features.html http://monetdb.cwi.nl/SQL/Documentation/EXPLAIN-Statement.html#EXPLAIN-Stat… > get histogram statistics i.e way to find that histogram exists on attribute or not through Xquery interface. not through sql . MonetDB does not rely on separate statistics tables. > Please reply as soon as possible. > > thank you. Actually i want the xquery construct equivalent to sql explain construct and so how it simulates histogram features i.e to get that percentage of query etc. Please reply as soon as possible. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

3 3

[Monetdb-developers] need help in monetdb xquery interface
by anshul gangwar 05 Mar '10

05 Mar '10

hello , I want to know whether there is any expression or command or anything equivalent to explain sql construct. Also What is the way to get histogram statistics i.e way to find that histogram exists on attribute or not through Xquery interface. not through sql . Please reply as soon as possible. thank you. The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

1 0

[Monetdb-developers] Announcement: New Feb2010 feature release of MonetDB suite
by Sjoerd Mullender 02 Mar '10

02 Mar '10

The MonetDB team at CWI/MonetDB BV is pleased to announce the Feb2010 feature release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Feb2010/>. -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, Feb2010, 1.337.2.33, 1.337.2.34
by Stefan Manegold 01 Mar '10

01 Mar '10

On Mon, Mar 01, 2010 at 10:20:01AM +0100, Sjoerd Mullender wrote: > Note that this particular initialization doesn't do anything. Static > variables are automatically initialized to 0. Thanks! --- I overlooked that one --- then the rather high value of hottop (600037897) I encountered was indeed only cause by incrementing it "unboundedly" rather than an "unexpected" initialization ... > Also note that in the HEAD branch, hottop is initialized to MAXHOT > (causing a propagation conflict). I indeed did not check the HEAD branch --- I cannot tell which is supposed to be the "correct" initial value, 0 or MAXHOT --- maybe Martin can help ... Stefan > On 2010-03-01 09:27, Stefan Manegold wrote: > > Update of /cvsroot/monetdb/MonetDB5/src/mal > > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv17757/MonetDB5/src/mal > > > > Modified Files: > > Tag: Feb2010 > > mal_interpreter.mx > > Log Message: > > > > make sure we initialize the static global variable hottop at least once; > > I hope, 0 is the correct initial value. > > > > > > Index: mal_interpreter.mx > > =================================================================== > > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > > retrieving revision 1.337.2.33 > > retrieving revision 1.337.2.34 > > diff -u -d -r1.337.2.33 -r1.337.2.34 > > --- mal_interpreter.mx 28 Feb 2010 18:49:24 -0000 1.337.2.33 > > +++ mal_interpreter.mx 1 Mar 2010 08:27:04 -0000 1.337.2.34 > > @@ -89,7 +89,7 @@ > > lng claim; /* actual claim on memory*/ > > int bid; > > } hotpotatoes[MAXHOT]; > > -static int hottop; > > +static int hottop = 0; > > @- > > The struct alignment leads to 40% gain in simple instructions when set. > > @c > > > > > > ------------------------------------------------------------------------------ > > Download Intel® Parallel Studio Eval > > Try the new software tools for yourself. Speed compiling, find bugs > > proactively, and fine-tune applications for parallel performance. > > See why Intel Parallel Studio got high marks during beta. > > http://p.sf.net/sfu/intel-sw-dev > > _______________________________________________ > > Monetdb-checkins mailing list > > Monetdb-checkins(a)lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > -- > Sjoerd Mullender > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

1 0

[Monetdb-developers] Mx segfaults during MonetDB build
by Mark Bucciarelli 01 Mar '10

01 Mar '10

Hi, I'm getting a segfault when running Mx, in buildtools/Mx/Code.c, CodeSubBlk(). The symptom is around line 305---s[-1] is not accessible memory. Any suggestions for how to debug/fix this? I got as far as looking at def in the parent frame (CodeSub()): (gdb) print *def $9 = { d_dir = Mxmacro, d_cmd = 0x8592e110 "DEFseqscan", d_blk = 0x89ca2000 "@:DEFseqscan_@[10](@1,@2,@3,@4,@5,@6,@7,@8,@9)@\n", d_mod = 0, d_sec = 0, d_cms = 0xd0d0d0d0 <Address 0xd0d0d0d0 out of bounds>, d_file = 0x86c3a080 "gdk_scanselect_defs.mx", d_line = 132 } (gdb) This is the Feb2010 branch, on OpenBSD 4.6 with the following malloc.conf options: A ``Abort''. malloc() will coredump the process, rather than tol- erate internal inconsistencies or incorrect usage. This is the default and a very handy debugging aid, since the core file rep- resents the time of failure, rather than when the bogus pointer was used. G ``Guard''. Enable guard pages. Each page size or larger alloca- tion is followed by a guard page that will cause a segmentation fault upon any access. J ``Junk''. Fill some junk into the area allocated. Currently junk is bytes of 0xd0 when allocating; this is pronounced ``Duh''. :-) Freed chunks are filled with 0xdf. P ``Move allocations within a page.'' Allocations larger than half a page but smaller than a page are aligned to the end of a page to catch buffer overruns in more cases. This is the default. I was getting these when building Nov2009, but I was able to get past them by restarting the build multiple times. I'd like to fix it this time around. Fortunately, after a gmake clean the crash is reproduceable. Thanks, m

2 1

[Monetdb-developers] pf:new-doc
by Tinco Andringa 26 Feb '10

26 Feb '10

Hello MonetDB developers, In my quest to develop an object oriented interface to monetdb/xquery on ruby using datamapper, I've stumbled upon the following problem. It is at this moment impossible to add a document to monetdb that does not have a source outside monetdb. The only function to add documents to monetdb seems to be pf:add-doc, which takes an url parameter with a location of an xml document it should import. To fully abstract the database from the ruby application, it would be nice if the developers could add a new (empty) document to the database. Would it be a nice idea to have the following function: pf:new-doc("document_name.xml"[, "collection_name.xml"[,"<some><initial>content</initial></some>"]]) It would make users of my api a little happier, and perhaps MonetDB/XQuery as a whole, a bit more normal ;) Kind regards, Tinco Andringa

1 1

[Monetdb-developers] SQL concurrency/serialization
by Isidor Zeuner 25 Feb '10

25 Feb '10

Dear MonetDB developers, on concurrently running writing transactions, I am getting a rather narrowly descriptive error message: "!SQLException:sql.trans:COMMIT: failed". My first attempts to deal with this issue suggest that it happens exactly in those cases where the concurrent writes create some sort of conflict. I am aware that MonetDB does not focus on concurrency, and of the suggestion to serialize multiple writers. However, I'd like to know what the current status of concurrency is. To what extent are SQL primitives implemented which allow to let the server take care of serializing writes, for example through explicit table locking or transaction isolation? Best regards, Isidor Zeuner

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_mapreduce.mx, , 1.13, 1.14
by Martin Kersten 24 Feb '10

24 Feb '10

Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv10474 > > Modified Files: > opt_mapreduce.mx > Log Message: > One of the optimisers 'inline,remap,evaluate,costModel,coercions,emptySet,aliases' replaces a nil statement in such a way that it ends up as argument 'nil' witout type, causing execution to fail. Push barrier up, such that we don't get undefined var errors because the reduce phase was performed in the dataflow block too. You can freeze the type of variables using setVarUDFtype. As a line of defense, at the end of an optimizer all non-UDFtypes are thrown away and a fresh type resolve step is taken.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/rdf rdf_shredder.mx, Feb2010, 1.13, 1.13.2.1
by Sjoerd Mullender 23 Feb '10

23 Feb '10

Are you sure *all* these changes are for disabling tokenizer? Including the property changes? None of this will now be propagated, so if any of this should land in the HEAD, you need to do it manually (but then please identically). On 2010-02-22 16:43, Lefteris Sidirourgos wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv32552/src/modules/mal/rdf > > Modified Files: > Tag: Feb2010 > rdf_shredder.mx > Log Message: > disable the tokenizer.mx from m5 > > > Index: rdf_shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf/rdf_shredder.mx,v > retrieving revision 1.13 > retrieving revision 1.13.2.1 > diff -u -d -r1.13 -r1.13.2.1 > --- rdf_shredder.mx 8 Jan 2010 12:08:58 -0000 1.13 > +++ rdf_shredder.mx 22 Feb 2010 15:43:41 -0000 1.13.2.1 > @@ -27,7 +27,9 @@ > #include "url.h" > #include "rdf.h" > #include "raptor.h" > +#if 0 > #include "../tokenizer.h" > +#endif > > typedef struct graphBATdef { > graphBATType batType; /* BAT type */ > @@ -36,6 +38,8 @@ > int tailType; /* type of right column */ > } graphBATdef; > > +static BUN batsz = 10000000; > + > /* this list should be kept alligned with the graphBATType enum */ > #if STORE == TRIPLE_STORE > static graphBATdef graphdef[N_GRAPH_BAT] = { > @@ -132,7 +136,8 @@ > @:rdf_BUNappend_unq(@1, @2)@ > #endif > > -@= rdf_BUNappend_unq > + > +@= rdf_BUNappend_unq_1 > bun = BUNfnd(BATmirror(@1),(ptr)@2); > if (bun == BUN_NONE) { > if (BATcount(@1) > 4 * @1->T->hash->mask) { > @@ -148,6 +153,20 @@ > bun = (@1)->hseqbase + bun; > } > > +@= rdf_BUNappend_unq > +bun = BUNfnd(BATmirror(@1),(ptr)@2); > +if (bun == BUN_NONE) { > + if (BATcount(@1) > 4 * @1->T->hash->mask) { > + HASHdestroy(@1); > + BAThash(BATmirror(@1), 2*BATcount(@1)); > + } > + bun = (BUN) @1->batCount; > + @1 = BUNappend(@1, (ptr)@2, TRUE); > + if (@1 == NULL) { > + @:raptor_exception(pdata, "could not append in@1")@ > + } > +} > + > @= rdf_tknzr_insert > { > str t = @1; > @@ -226,12 +245,9 @@ > > /* disable all properties */ > b->tsorted = FALSE; > - b->T->nosorted = 0; > b->tdense = FALSE; > - b->T->nodense = 0; > b->tkey = FALSE; > - b->T->nokey[0] = 0; > - b->T->nokey[1] = 1; > + b->hdense = TRUE; > > return b; > } > @@ -264,7 +280,7 @@ > pdata->graph[i] = create_BAT ( > graphdef[i].headType, > graphdef[i].tailType, > - BATTINY); /* DOTO: estimate size */ > + batsz); /* DOTO: estimate size */ > if (pdata->graph[i] == NULL) { > return NULL; > } > @@ -274,7 +290,7 @@ > pdata->graph[MAP_LEX] = create_BAT ( > graphdef[MAP_LEX].headType, > graphdef[MAP_LEX].tailType, > - BATTINY); /* DOTO: estimate size */ > + batsz); /* DOTO: estimate size */ > if (pdata->graph[MAP_LEX] == NULL) { > return NULL; > } > @@ -346,10 +362,10 @@ > static str > post_processing (parserData *pdata) > { > -#if STORE == TRIPLE_STORE > BUN cnt; > BAT *map_oid = NULL, *S = NULL, *P = NULL, *O = NULL, *ctref= NULL; > BAT **graph = pdata->graph; > +#ifdef _TKNZR_H > BATiter bi, mi; > BUN p, d, r; > oid *bt; > @@ -379,7 +395,31 @@ > P = graph[P_sort]; > O = graph[O_sort]; > cnt = BATcount(S); > +#else > + /* order MAP_LEX */ > + BATorder(BATmirror(graph[MAP_LEX])); > + map_oid = BATmark(graph[MAP_LEX], 0); /* BATmark will create a copy */ > + BATorder(map_oid); > + BATsetaccess(map_oid, BAT_READ); /* force BAtmark not to copy bat */ > + map_oid = BATmirror(BATmark(BATmirror(map_oid), 0)); > + BATsetaccess(graph[MAP_LEX], BAT_READ); /* force BATmark not to copy bat */ > + graph[MAP_LEX] = BATmirror(BATmark(BATmirror(graph[MAP_LEX]), 0)); > + > + /* convert old oids of S_sort, P_sort, O_sort to new ones */ > + cnt = BATcount(graph[S_sort]); > + S = BATleftfetchjoin(graph[S_sort], map_oid, cnt); > + if (S == NULL) goto bailout; > + BBPreclaim(graph[S_sort]); > + P = BATleftfetchjoin(graph[P_sort], map_oid, cnt); > + if (P == NULL) goto bailout; > + BBPreclaim(graph[P_sort]); > + O = BATleftfetchjoin(graph[O_sort], map_oid, cnt); > + if (O == NULL) goto bailout; > + BBPreclaim(graph[O_sort]); > + BBPreclaim(map_oid); > +#endif > > +#if STORE == TRIPLE_STORE > /* order SPO/SOP */ > graph[S_sort] = BATmirror(BATsort(BATmirror(S))); /* sort on S */ > @:order(graph[S_sort],P,O,PO)@ > @@ -408,6 +448,14 @@ > > return MAL_SUCCEED; > > +#elif STORE == MLA_STORE > + graph[S_sort] = S; > + graph[P_sort] = P; > + graph[O_sort] = O; > + > + return MAL_SUCCEED; > +#endif > + > bailout: > if (map_oid != NULL) BBPreclaim(map_oid); > if (ctref != NULL) BBPreclaim(ctref); > @@ -415,11 +463,6 @@ > if (P != NULL) BBPreclaim(P); > if (O != NULL) BBPreclaim(O); > return NULL; > - > -#elif STORE == MLA_STORE > - (void) pdata; > - return MAL_SUCCEED; > -#endif > } > > @= clean_raptor > @@ -455,15 +498,21 @@ > (void) graphname; > > /* init tokenizer */ > +#ifdef _TKNZR_H > if (TKNZRopen (NULL, schema) != MAL_SUCCEED) { > throw(RDF, "rdf.rdfShred", > "could not open the tokenizer\n"); > } > +#else > + (void) schema; > +#endif > > /* Init pdata */ > pdata = parserData_create(*location); > if (pdata == NULL) { > +#ifdef _TKNZR_H > TKNZRclose(&iret); > +#endif > @:clean@ > throw(RDF, "rdf.rdfShred", > "could not allocate enough memory for pdata\n"); > @@ -473,7 +522,9 @@ > raptor_init(); > pdata->rparser = rparser = raptor_new_parser("guess"); > if (rparser == NULL) { > +#ifdef _TKNZR_H > TKNZRclose(&iret); > +#endif > raptor_finish(); > @:clean@ > throw(RDF, "rdf.rdfShred", "could not create raptor parser object\n"); > @@ -484,7 +535,9 @@ > /* Parse URI or local file. */ > ret = URLisaURL(&isURI, location); > if (ret != MAL_SUCCEED) { > +#ifdef _TKNZR_H > TKNZRclose(&iret); > +#endif > @:clean@ > return ret; > } else if (isURI) { > @@ -496,6 +549,9 @@ > iret = raptor_parse_file(rparser, uri, NULL); > } > @:clean_raptor@ > +#ifdef _TKNZR_H > + TKNZRclose(&iret); > +#endif > > graph = pdata->graph; > assert (pdata->tcount == BATcount(graph[S_sort]) && > @@ -504,23 +560,18 @@ > > /* error check */ > if (iret) { > - TKNZRclose(&iret); > @:clean@ > throw(RDF, "rdf.rdfShred", "parsing failed\n"); > } > if (pdata->exception) { > - TKNZRclose(&iret); > throw(RDF, "rdf.rdfShred", "%s\n", pdata->exceptionMsg); > } else if (pdata->fatal) { > - TKNZRclose(&iret); > throw(RDF, "rdf.rdfShred", "last fatal error was:\n%s\n", > pdata->fatalMsg); > } else if (pdata->error) { > - TKNZRclose(&iret); > throw(RDF, "rdf.rdfShred", "last error was:\n%s\n", > pdata->errorMsg); > } else if (pdata->warning) { > - TKNZRclose(&iret); > throw(RDF, "rdf.rdfShred", "last warning was:\n%s\n", > pdata->warningMsg); > } > @@ -528,7 +579,6 @@ > /* post processing step */ > ret = post_processing(pdata); > if (ret != MAL_SUCCEED) { > - TKNZRclose(&iret); > @:clean@ > throw(RDF, "rdf.rdfShred", "could not post-proccess data"); > } > @@ -536,7 +586,6 @@ > /* prepare return bat of bats */ > retbat = BATnew(TYPE_void, TYPE_bat, N_GRAPH_BAT); > if (retbat == NULL) { > - TKNZRclose(&iret); > @:clean@ > throw(RDF, "rdf.rdfShred", > "could not allocate enough memory for return bat"); > @@ -548,7 +597,6 @@ > } > > GDKfree(pdata); > - TKNZRclose(&iret); > BBPkeepref(*retval = retbat->batCacheid); > > return MAL_SUCCEED; > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/rdf rdf_shredder.mx, Feb2010, 1.13.2.1, 1.13.2.2
by Sjoerd Mullender 22 Feb '10

22 Feb '10

Propagation prevented. On 2010-02-22 17:24, Lefteris Sidirourgos wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv8115/rdf > > Modified Files: > Tag: Feb2010 > rdf_shredder.mx > Log Message: > remove any files and code related to the tokenizer from the stable branch. These changes should not be propagated to the head > > > Index: rdf_shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf/rdf_shredder.mx,v > retrieving revision 1.13.2.1 > retrieving revision 1.13.2.2 > diff -u -d -r1.13.2.1 -r1.13.2.2 > --- rdf_shredder.mx 22 Feb 2010 15:43:41 -0000 1.13.2.1 > +++ rdf_shredder.mx 22 Feb 2010 16:24:23 -0000 1.13.2.2 > @@ -27,9 +27,6 @@ > #include "url.h" > #include "rdf.h" > #include "raptor.h" > -#if 0 > -#include "../tokenizer.h" > -#endif > > typedef struct graphBATdef { > graphBATType batType; /* BAT type */ > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0