Subject: RE: Monet DB for RDF databases, BLOB
From: James McGlothlin <[email-masked]>
To: monetdb-martin <[email-masked]>
Date: Thu, 26 Feb 2009 00:13:24 -0600

If possible I need you to answer one more question for me.  I have my project working with MonetDb and so far my performance results are good.  But I have a need to store a BLOB (unless they is a better way to store a long bit array).

Java JDBC defines BLOB as an interface class that cannot be instantiated.  All of the major databases like SQL Server and Oracle show a class definition (for example import oracle.java.blob and instantiate BLOB) .  I cannot find any information on how I can instantiate a BLOB in Monet DB or otherwise write an outputstream to a tuple.

Thanks again for your help.  This is rather urgent since I am trying to submit a paper for VLDB. It should be beneficial to MonetDB as I am hoping to report results using a MonetDB system.  

James P. McGlothlin
Research Assistant
The University of Texas At Dallas

> Date: Fri, 13 Feb 2009 17:19:46 +0100
> From: [email-masked]
> To: [email-masked]
> Subject: Re: Monet DB for RDF databases
> 
> James McGlothlin wrote:
> > Thank you for your quick response. 
> > 
> > I had actually carefully reviewed the paper you referenced me to.  That 
> > is part of what sent me to MonetDB in the first place.
> > 
> > I have one more question for you at this time.  I need to store bit 
> > matrices in a table.  They will be rather large and 99% zeroes.
> You may have a look at FastBits of A. SHoshani of Lawrence livermore,
> who implemented bitindexing using MonetDB
> 
> > Is the BLOB type the most appropriate SQL type definition?  Does MonetDB 
> > support compression itself or do I need to execute run length encoding 
> > prior to inserting the BLOB?  I see other databases that use LZO 
> > compression within the database, and I would rather keep the logic lower 
> > than write it on top, as you suggest.
> There are too many answers to this simple question.
> > 
> > Thanks again for your help,,
> > 
> > James P. McGlothlin
> > Research Assistant
> > The University of Texas At Dallas
> > 
> > 
> >  > Date: Thu, 12 Feb 2009 19:30:46 +0100
> >  > From: [email-masked]
> >  > To: [email-masked]
> >  > CC: [email-masked]
> >  > Subject: Re: Monet DB for RDF databases
> >  >
> >  > James McGlothlin wrote:
> >  > > I am currently developing a research project designed to increase the
> >  > > performance and scalability of RDF databases through a novel way of
> >  > > reorganizing the tables and queries.
> >  > You might have a look at our paper in VLDB08 on this topic
> >  > 
> > http://www.google.nl/url?sa=t&source=web&ct=res&cd=4&url=http%3A%2F%2Fwww.cwi.nl%2Fhtbin%2Fins1%2Fpublications%3Frequest%3Dpdf%26key%3DSiGoKeNeMa%3AVLDB%3A08&ei=Q2mUSd-_FsaC-gbs_oj9CA&usg=AFQjCNFkN194Duf0WFFz-enWm0dMk7BDjA&sig2=puLWaupj10F6XV8_hFJNMQ
> >  > >
> >  > > Based on research papers, I planned to implement my project utilizing
> >  > > column stores. Since CStore is no longer supported, and Monet DB is
> >  > > widely used, including in many research papers on triples stores and
> >  > > vertical partitioning, I planned to implement it with Monet DB. My 
> > plan
> >  > > was to utilize JDBC to create a solution that would be interoperatable
> >  > > if I needed to change to a traditional relational database or another
> >  > > product.
> >  > >
> >  > > My concern is that Monet DB does not support cursors. According to 
> > your
> >  > > documentation, this is because Monet DB does not support the 
> > iterator model.
> >  > >
> >  > > If I have a table containing 100million tuples, and I need to iterate
> >  > > over all data in this table, how can I do this efficiently? I would
> >  > use relational expressions to do the work INSIDE the database rather
> >  > then outside using cursors. Cursors and JDBC is most likely a dead end
> >  > for any serious implementation.
> >  > >presume that if I simply performa selecct statement it is not going to
> >  > > work being as the number of records is going to vastly outsize the
> >  > > available RAM.
> >  > The chop it into smaller selects, but avoid single tuples operations
> >  > over a channel
> >  > >
> >  > > Basically I need to iterate over all data in a table and perform 
> > actions
> >  > > on each record. As I only need to access one record at a time, it does
> >  > delegate logic towards the engine instead of pulling data out
> >  > > not seem that I should have a memory constraint issue. However, I am
> >  > > not certain how I could implement this in Monet DB. I would like to
> >  > > avoid using MAL so that I have a solution that can easily be ported to
> >  > > other databases.
> >  > regards, Martin
> >  > >
> >  > > Thanks for your assistance and advice,
> >  > > James McGlothlin
> >  > > Research Assistant
> >  > > The University Of Texas at Dallas
> >  >
>