We are running into an interesting problem that we can’t seem to resolve. We have a large number of xml files that we are parsing and loading to MonetDB Feb2009-SP2 sql server running on Redhat with 40gigs of memory. Basically, everything looks fine while the files are being loaded. But, once the server is shutdown and restarted (after the data load has been finished), all the data is simply gone! We have done a dump and restored the db before restarting the db, then loaded more data on top of it; and the data that was restored is intact. But, the newly loaded data is simply gone. It doesn’t seem to matter how small or large the data set being loaded is (tried as small as 1.5 gig of xml files resulting in 1.2M rows). Which leads me to believe our data is not persistent! (a minor problem.. J)

We also noticed that as our parser (java app using JDBC with hibernate) is loading the data, that the memory use continues to grow and never shrinks after the data load is complete. So, it seems like the all the data being loaded is being kept in the memory; but never becomes persistent. We tried waiting 4 hours after loading 1.2M rows, thinking that there may be a lag in syncing to the disk (which seem odd, in case of a sudden server failure); but the data simply is gone. Can you provide some assistance in troubleshooting this problem? Thanks.