On Thu, May 24, 2007 at 06:15:43PM +0200, Maarten Clements wrote:
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: 24 May 2007 18:00 To: M.Clements@cwi.nl Cc: monetdb-users@lists.sourceforge.net Subject: Re: [MonetDB-users] MonetDb-XQuery problem
On Thu, May 24, 2007 at 05:07:03PM +0200, Maarten Clements wrote:
Using the following XQuery in MonetDb (WinXP) crashes on large documents
the usual questions:
1) which version of MonetDB XQuery are you using? The latest, from sjoerd's directory
so I guess, this is MonetDB 4.17.1 / MonetDB/XQuery 0.17.1, right?
2) what exactly does "crash" mean? does the Mserver simply stop? Is the any error message? Does Mserver grow/excessively use resources before crashing? ...? It fills up my memory and after doing that Mserver stops.
Ok.
3) what does "large" and "small" mean wrt., your document sizes? (serialized file size in byte? number of /collection/doc nodes in either document? ...?) The problem occurs with C1 = 59Mb, C2 = 7Kb (but I didn't search for the critical sizes:) Both files contain movie data: C1 is IMDB data with nodes: title,year,tag1,tag2...tagx (im not sure how many movies) C2 is Netflix data with nodes: title, year (100 movies)
count(doc("C1.xml")/collection/doc) ? count(doc("C2.xml")/collection/doc) ? for both the size that wokr for you, and that that does not (just one for each, no need to find the switching point)
4) how much memory does your machine have? 1 Gb
(when C1 and C2 are small it works ok):
<col> { for $b in doc("C1.xml")/collection/doc for $top in doc("C2.xml")/collection/doc where ($b/year > 1950) and ($b/title = $top/title) return <movie> {$b/year} </movie> } </col>
If I remove one of the two conditions it works perfectly, e.g.:
<col> { for $b in doc("C1.xml")/collection/doc for $top in doc("C2.xml")/collection/doc where ($b/title = $top/title) return <movie> {$b/year} </movie> } </col>
Can anyone explain this problem?
most probably, the more complex join condition is not recognized as join, then the intermediate result (corss product) blows up.
It is very likely that this is indeed the problem... Why does this happen for a query this simple?
simply because join recognition in XQuery is not "simple" at all. But you can try the algebra version, instead --- well, assuming that you won't/can't recompile on Windows, you need to use the pf compiler "by hand", from a shell 9"command prompt") like : pf -A query.xq | Mserver <options> or, with a MonetDB/XQuery server running: pf -A query.xq | MapiClient -lmil <options> However, I cannot tell you, which <options> you need to sucessfully run Mserver and/or MapiClient "by hand" from a shell on Windows ... Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |