
23 Dec
2013
23 Dec
'13
10:17 a.m.
I am trying to bulk-load a list of objects into a one-column (primary key) db. The only reason is to remove duplicates. I can't load the list in memory, because the file size is way greater than my memory size. I use monetdb's COPY-INTO<http://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInto>command, but I don't want it to fail when there is a duplicate. I want it to add everything that is not a duplicate and skip the duplicates. Is there any way to do that with monetdb? Any other way? Thank you in advance!