Crashed DB - BBPextend: trying to extend BAT pool beyond the limit
Hi, We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database: 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend
BAT pool beyond the limit (16384000) 2013-10-07 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has exited with exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database 'testbd' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25)
The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users. I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue: 1 - Is there any way to see a more detailed log about the error? 2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success. 3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have? Thank you, Pedro Salgueiro
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The message means that you are creating too many BATs, probably because you're using too many tables with too many columns. The number of BATs is a hard limit, but it can easily be extended with a recompilation of MonetDB. Increase one of the numbers N_BBPINIT or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, rethink your database schema and use fewer tables and columns. On 2013-10-07 13:02, Pedro Salgueiro wrote:
Hi,
We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database:
2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend BAT pool beyond the limit (16384000) 2013-10-07 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has exited with exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database 'testbd' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25)
The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users.
I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue:
1 - Is there any way to see a more detailed log about the error?
2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success.
3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have?
Thank you, Pedro Salgueiro
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUlKuqj7g04AjvIQpAQJB7wP/bMsajBiYpkZXmUSdgUsv3xzPHS/SGPB6 Rtd8h+bOjzfZREI8od7VRHlNmBnawPZbcfJiAaWovRNQvezRO4HZi5PKB73YW2ju IAJKS2aaUcln+R8D/RgmKcVYrxWLFFWcHbWciMw6LnKNUMHWObhmTnY7Ne1BSh1V Db7vlqZnkh8= =vuct -----END PGP SIGNATURE-----
I have already to changed the value of both variables, on the Feb2013-SP4 version. Now I get a different error when the client tries to connect to the same database: 2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' (8278) has
exited with exit status 15 2013-10-07 14:35:46 ERR merovingian[8219]: database 'testbd' crashed after SIGTERM 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database 'testbd' did not initialise the sql scenario
Is this a completely different error or is it related with the initial problem? I wonder if the database got corrupted when the initial problem occurred. If the database got corrupted, is there any way to get it back to normal operation? Also, what are the implications of changing these two variables? Would this increase the memory consumption or harm the performance of MonetDB in some way? Thank you, Pedro Salgueiro On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
The message means that you are creating too many BATs, probably because you're using too many tables with too many columns. The number of BATs is a hard limit, but it can easily be extended with a recompilation of MonetDB. Increase one of the numbers N_BBPINIT or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, rethink your database schema and use fewer tables and columns.
On 2013-10-07 13:02, Pedro Salgueiro wrote:
Hi,
We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database:
2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend BAT pool beyond the limit (16384000) 2013-10-07 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has exited with exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database 'testbd' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25)
The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users.
I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue:
1 - Is there any way to see a more detailed log about the error?
2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success.
3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have?
Thank you, Pedro Salgueiro
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlKuqj7g04AjvIQpAQJB7wP/bMsajBiYpkZXmUSdgUsv3xzPHS/SGPB6 Rtd8h+bOjzfZREI8od7VRHlNmBnawPZbcfJiAaWovRNQvezRO4HZi5PKB73YW2ju IAJKS2aaUcln+R8D/RgmKcVYrxWLFFWcHbWciMw6LnKNUMHWObhmTnY7Ne1BSh1V Db7vlqZnkh8= =vuct -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not. On 2013-10-07 15:57, Pedro Salgueiro wrote:
I have already to changed the value of both variables, on the Feb2013-SP4 version. Now I get a different error when the client tries to connect to the same database:
2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR merovingian[8219]: database 'testbd' crashed after SIGTERM 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database 'testbd' did not initialise the sql scenario
Is this a completely different error or is it related with the initial problem?
I wonder if the database got corrupted when the initial problem occurred. If the database got corrupted, is there any way to get it back to normal operation?
Also, what are the implications of changing these two variables? Would this increase the memory consumption or harm the performance of MonetDB in some way?
Thank you, Pedro Salgueiro
On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org>> wrote:
The message means that you are creating too many BATs, probably because you're using too many tables with too many columns. The number of BATs is a hard limit, but it can easily be extended with a recompilation of MonetDB. Increase one of the numbers N_BBPINIT or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, rethink your database schema and use fewer tables and columns.
On 2013-10-07 13:02, Pedro Salgueiro wrote:
Hi,
We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database:
2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend BAT pool beyond the limit (16384000) 2013-10-07 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has exited with exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database 'testbd' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25)
The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users.
I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue:
1 - Is there any way to see a more detailed log about the error?
2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success.
3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have?
Thank you, Pedro Salgueiro
_______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUlLYwD7g04AjvIQpAQIXuwQAqsq/1uEzVi2UNXSFOBC+ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/8UgE4P68MzAC23HusltxM6AZYhD4uFAJY3 /SzLNEBlKF2C3BqnIKyWlSqjtOwOArcRJC5RdGSd7Fj/48UVCrR0HtWVqhT4/O9Q LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE-----
I changed the variables one at a time, never both at the same time. Still, I get the other error while connecting to the same DB. I used the source obtained via "apt-get source monetdb5-server monetdb5-sql monetdb-client". After changing the code, I cleaned up everything with "make clean", then I recompiled everything using "dpkg-buildpackage" which build the new packages. I guess this should recompile everything, right? I might try a new build from the source tarball to see if I get the same behavior with this DB. Best regards, Pedro Salgueiro On Mon, Oct 7, 2013 at 4:52 PM, Sjoerd Mullender <sjoerd@acm.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not.
On 2013-10-07 15:57, Pedro Salgueiro wrote:
I have already to changed the value of both variables, on the Feb2013-SP4 version. Now I get a different error when the client tries to connect to the same database:
2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR merovingian[8219]: database 'testbd' crashed after SIGTERM 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database 'testbd' did not initialise the sql scenario
Is this a completely different error or is it related with the initial problem?
I wonder if the database got corrupted when the initial problem occurred. If the database got corrupted, is there any way to get it back to normal operation?
Also, what are the implications of changing these two variables? Would this increase the memory consumption or harm the performance of MonetDB in some way?
Thank you, Pedro Salgueiro
On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org>> wrote:
The message means that you are creating too many BATs, probably because you're using too many tables with too many columns. The number of BATs is a hard limit, but it can easily be extended with a recompilation of MonetDB. Increase one of the numbers N_BBPINIT or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, rethink your database schema and use fewer tables and columns.
On 2013-10-07 13:02, Pedro Salgueiro wrote:
Hi,
We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database:
2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend BAT pool beyond the limit (16384000) 2013-10-07 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has exited with exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database 'testbd' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25)
The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users.
I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue:
1 - Is there any way to see a more detailed log about the error?
2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success.
3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have?
Thank you, Pedro Salgueiro
_______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlLYwD7g04AjvIQpAQIXuwQAqsq/1uEzVi2UNXSFOBC+ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/8UgE4P68MzAC23HusltxM6AZYhD4uFAJY3 /SzLNEBlKF2C3BqnIKyWlSqjtOwOArcRJC5RdGSd7Fj/48UVCrR0HtWVqhT4/O9Q LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Perhaps you can share more information about your application. Is it product we can refer too? Can you give an url? Having a database grow beyond 16M columns is indicative of some series application design issues, e.g. proper garbage collection of spurious tables left behind by your application as persistent. Given your indicative size of the database (1.1G) it appears that otherwise columns will be in the order of a few MB each. regards, Martin On 10/7/13 7:23 PM, Pedro Salgueiro wrote:
I changed the variables one at a time, never both at the same time. Still, I get the other error while connecting to the same DB.
I used the source obtained via "apt-get source monetdb5-server monetdb5-sql monetdb-client". After changing the code, I cleaned up everything with "make clean", then I recompiled everything using "dpkg-buildpackage" which build the new packages. I guess this should recompile everything, right?
I might try a new build from the source tarball to see if I get the same behavior with this DB.
Best regards, Pedro Salgueiro
On Mon, Oct 7, 2013 at 4:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org>> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not.
On 2013-10-07 15:57, Pedro Salgueiro wrote: > I have already to changed the value of both variables, on the > Feb2013-SP4 version. Now I get a different error when the client > tries to connect to the same database: > > 2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' > (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR > merovingian[8219]: database 'testbd' crashed after SIGTERM > 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database > 'testbd' did not initialise the sql scenario > > > Is this a completely different error or is it related with the > initial problem? > > I wonder if the database got corrupted when the initial problem > occurred. If the database got corrupted, is there any way to get it > back to normal operation? > > Also, what are the implications of changing these two variables? > Would this increase the memory consumption or harm the performance > of MonetDB in some way? > > Thank you, Pedro Salgueiro > > > > On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org> > <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>>> wrote: > > The message means that you are creating too many BATs, probably > because you're using too many tables with too many columns. The > number of BATs is a hard limit, but it can easily be extended with > a recompilation of MonetDB. Increase one of the numbers N_BBPINIT > or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, > rethink your database schema and use fewer tables and columns. > > On 2013-10-07 13:02, Pedro Salgueiro wrote: >> Hi, > >> We have a MonetDB instance which was working without problems >> and suddenly stopped working. The following error shows up in >> "merovingian.log" when the client(mclient) tries to connect to >> the database: > >> 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying >> to extend BAT pool beyond the limit (16384000) 2013-10-07 >> 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG >> merovingian[12015]: database 'testbd' (12111) has exited with >> exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client >> error: database 'testbd' has crashed after starting, manual >> intervention needed, check monetdbd's logfile for details >> 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has >> crashed after start on 2013-10-02 16:36:06, attempting restart, >> up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 >> (25-0=25) > > >> The DB size is about 1.1 GB. I really don't know how many >> tables, columns or records this specific database has, as it was >> being used for a while and the data is created/uploaded daily by >> its users. > >> I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a >> few question about this issue: > >> 1 - Is there any way to see a more detailed log about the error? > >> 2 - Is it possible to recover the DB from this error? I saw >> somewhere in the mailing list that removing the sql_logs could >> help starting the DB, at the cost of loosing some data. Already >> tried that with no success. > >> 3 - What is this "BAT pool limit" thing? Is this related in >> some way with the number of tables, number of columns per table, >> or total amount of columns that a DB can have? > >> Thank you, Pedro Salgueiro > > > > > >> _______________________________________________ users-list >> mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> >> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> >> https://www.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > https://www.monetdb.org/mailman/listinfo/users-list > > > > > _______________________________________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org> > https://www.monetdb.org/mailman/listinfo/users-list >
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlLYwD7g04AjvIQpAQIXuwQAqsq/1uEzVi2UNXSFOBC+ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/8UgE4P68MzAC23HusltxM6AZYhD4uFAJY3 /SzLNEBlKF2C3BqnIKyWlSqjtOwOArcRJC5RdGSd7Fj/48UVCrR0HtWVqhT4/O9Q LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Unfortunately we can't provider specific details about the application. Basically, it create tables on-demand, some of them(very few) may have at most a few hundreds of columns. Among these tables, some are temporary and a very short life-time, being dropped when they are no longer needed. We really had a bug which was preventing some temporary tables from being deleted. Still, I really doubt that we have reached the limit of 16M columns. This is the main reason why we wanted to recover the database, so we can understand what the problem was. We are dropping the temporary tables with a simple "drop table" sql statement. Could this method of dropping the tables leave some "garbage" behind, affecting the number of columns seen by Monet, even when the tables has already been deleted? Is there the need of doing some cleanup after dropping a table? If so, what should we do? Best regards, Pedro On Mon, Oct 7, 2013 at 8:16 PM, Martin Kersten <Martin.Kersten@cwi.nl>wrote:
Hi
Perhaps you can share more information about your application. Is it product we can refer too? Can you give an url?
Having a database grow beyond 16M columns is indicative of some series application design issues, e.g. proper garbage collection of spurious tables left behind by your application as persistent. Given your indicative size of the database (1.1G) it appears that otherwise columns will be in the order of a few MB each.
regards, Martin
On 10/7/13 7:23 PM, Pedro Salgueiro wrote:
I changed the variables one at a time, never both at the same time. Still, I get the other error while connecting to the same DB.
I used the source obtained via "apt-get source monetdb5-server monetdb5-sql monetdb-client". After changing the code, I cleaned up everything with "make clean", then I recompiled everything using "dpkg-buildpackage" which build the new packages. I guess this should recompile everything, right?
I might try a new build from the source tarball to see if I get the same behavior with this DB.
Best regards, Pedro Salgueiro
On Mon, Oct 7, 2013 at 4:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto: sjoerd@acm.org>> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not.
On 2013-10-07 15:57, Pedro Salgueiro wrote: > I have already to changed the value of both variables, on the > Feb2013-SP4 version. Now I get a different error when the client > tries to connect to the same database: > > 2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' > (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR > merovingian[8219]: database 'testbd' crashed after SIGTERM > 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database > 'testbd' did not initialise the sql scenario > > > Is this a completely different error or is it related with the > initial problem? > > I wonder if the database got corrupted when the initial problem > occurred. If the database got corrupted, is there any way to get it > back to normal operation? > > Also, what are the implications of changing these two variables? > Would this increase the memory consumption or harm the performance > of MonetDB in some way? > > Thank you, Pedro Salgueiro > > > > On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org<mailto: sjoerd@acm.org> > <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>>> wrote: > > The message means that you are creating too many BATs, probably > because you're using too many tables with too many columns. The > number of BATs is a hard limit, but it can easily be extended with > a recompilation of MonetDB. Increase one of the numbers N_BBPINIT > or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, > rethink your database schema and use fewer tables and columns. > > On 2013-10-07 13:02, Pedro Salgueiro wrote: >> Hi, > >> We have a MonetDB instance which was working without problems >> and suddenly stopped working. The following error shows up in >> "merovingian.log" when the client(mclient) tries to connect to >> the database: > >> 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying >> to extend BAT pool beyond the limit (16384000) 2013-10-07 >> 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG >> merovingian[12015]: database 'testbd' (12111) has exited with >> exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client >> error: database 'testbd' has crashed after starting, manual >> intervention needed, check monetdbd's logfile for details >> 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has >> crashed after start on 2013-10-02 16:36:06, attempting restart, >> up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 >> (25-0=25) > > >> The DB size is about 1.1 GB. I really don't know how many >> tables, columns or records this specific database has, as it was >> being used for a while and the data is created/uploaded daily by >> its users. > >> I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a >> few question about this issue: > >> 1 - Is there any way to see a more detailed log about the error? > >> 2 - Is it possible to recover the DB from this error? I saw >> somewhere in the mailing list that removing the sql_logs could >> help starting the DB, at the cost of loosing some data. Already >> tried that with no success. > >> 3 - What is this "BAT pool limit" thing? Is this related in >> some way with the number of tables, number of columns per table, >> or total amount of columns that a DB can have? > >> Thank you, Pedro Salgueiro > > > > > >> ______________________________**_________________ users-list >> mailing list users-list@monetdb.org <mailto: users-list@monetdb.org**> >> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org**>>
>> https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list> > > > > ______________________________**_________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org**>>
> https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list> > > > > > ______________________________**_________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org**> > https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list> >
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlLYwD7g04AjvIQpAQIXuw**QAqsq/1uEzVi2UNXSFOBC+**ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/**8UgE4P68MzAC23HusltxM6AZYhD4uF**AJY3 /**SzLNEBlKF2C3BqnIKyWlSqjtOwOArc**RJC5RdGSd7Fj/48UVCrR0HtWVqhT4/** O9Q LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE----- ______________________________**_________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org**> https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**_________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**_________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
On 10/8/13 12:16 PM, Pedro Salgueiro wrote:
Unfortunately we can't provider specific details about the application.
Basically, it create tables on-demand, some of them(very few) may have at most a few hundreds of columns. Among these tables, some are temporary and a very short life-time, being dropped when they are no longer needed. We really had a bug which was preventing some temporary tables from being deleted. Still, I really doubt that we have reached the limit of 16M columns. This is the main reason why we wanted to recover the database, so we can understand what the problem was.
We are dropping the temporary tables with a simple "drop table" sql statement. Could this method of dropping the tables leave some "garbage" behind, affecting the number of columns seen by Monet, even when the tables has already been deleted? Is there the need of doing some cleanup after dropping a table? If so, what should we do? If you are expecting left-over garbage from the application, then look into the SQL catalog tables for their information. If you expect low level garbage, go and look in the growth rate of DBfarm/bat* directory. regards, Martin
Best regards, Pedro
On Mon, Oct 7, 2013 at 8:16 PM, Martin Kersten <Martin.Kersten@cwi.nl <mailto:Martin.Kersten@cwi.nl>> wrote:
Hi
Perhaps you can share more information about your application. Is it product we can refer too? Can you give an url?
Having a database grow beyond 16M columns is indicative of some series application design issues, e.g. proper garbage collection of spurious tables left behind by your application as persistent. Given your indicative size of the database (1.1G) it appears that otherwise columns will be in the order of a few MB each.
regards, Martin
On 10/7/13 7:23 PM, Pedro Salgueiro wrote:
I changed the variables one at a time, never both at the same time. Still, I get the other error while connecting to the same DB.
I used the source obtained via "apt-get source monetdb5-server monetdb5-sql monetdb-client". After changing the code, I cleaned up everything with "make clean", then I recompiled everything using "dpkg-buildpackage" which build the new packages. I guess this should recompile everything, right?
I might try a new build from the source tarball to see if I get the same behavior with this DB.
Best regards, Pedro Salgueiro
On Mon, Oct 7, 2013 at 4:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org> <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>>> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not.
On 2013-10-07 15:57, Pedro Salgueiro wrote: > I have already to changed the value of both variables, on the > Feb2013-SP4 version. Now I get a different error when the client > tries to connect to the same database: > > 2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' > (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR > merovingian[8219]: database 'testbd' crashed after SIGTERM > 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database > 'testbd' did not initialise the sql scenario > > > Is this a completely different error or is it related with the > initial problem? > > I wonder if the database got corrupted when the initial problem > occurred. If the database got corrupted, is there any way to get it > back to normal operation? > > Also, what are the implications of changing these two variables? > Would this increase the memory consumption or harm the performance > of MonetDB in some way? > > Thank you, Pedro Salgueiro > > > > On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender <sjoerd@acm.org <mailto:sjoerd@acm.org> <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>> > <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org> <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>>>> wrote: > > The message means that you are creating too many BATs, probably > because you're using too many tables with too many columns. The > number of BATs is a hard limit, but it can easily be extended with > a recompilation of MonetDB. Increase one of the numbers N_BBPINIT > or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, > rethink your database schema and use fewer tables and columns. > > On 2013-10-07 13:02, Pedro Salgueiro wrote: >> Hi, > >> We have a MonetDB instance which was working without problems >> and suddenly stopped working. The following error shows up in >> "merovingian.log" when the client(mclient) tries to connect to >> the database: > >> 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying >> to extend BAT pool beyond the limit (16384000) 2013-10-07 >> 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG >> merovingian[12015]: database 'testbd' (12111) has exited with >> exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client >> error: database 'testbd' has crashed after starting, manual >> intervention needed, check monetdbd's logfile for details >> 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has >> crashed after start on 2013-10-02 16:36:06, attempting restart, >> up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 >> (25-0=25) > > >> The DB size is about 1.1 GB. I really don't know how many >> tables, columns or records this specific database has, as it was >> being used for a while and the data is created/uploaded daily by >> its users. > >> I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a >> few question about this issue: > >> 1 - Is there any way to see a more detailed log about the error? > >> 2 - Is it possible to recover the DB from this error? I saw >> somewhere in the mailing list that removing the sql_logs could >> help starting the DB, at the cost of loosing some data. Already >> tried that with no success. > >> 3 - What is this "BAT pool limit" thing? Is this related in >> some way with the number of tables, number of columns per table, >> or total amount of columns that a DB can have? > >> Thank you, Pedro Salgueiro > > > > > >> _________________________________________________ users-list >> mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__> >> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__>>
>> https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list> > > > > _________________________________________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__>>
> https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list> > > > > > _________________________________________________ users-list mailing > list users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__> > https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list> >
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlLYwD7g04AjvIQpAQIXuw__QAqsq/1uEzVi2UNXSFOBC+__ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/__8UgE4P68MzAC23HusltxM6AZYhD4uF__AJY3 /__SzLNEBlKF2C3BqnIKyWlSqjtOwOArc__RJC5RdGSd7Fj/48UVCrR0HtWVqhT4/__O9Q LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE----- _________________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>__> https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list>
_________________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list>
_________________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/__mailman/listinfo/users-list <https://www.monetdb.org/mailman/listinfo/users-list>
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Another question: will a DB created on Windows run in Linux? This could be related to the error that I am seeing after recompiling Monet, since the original DB was created on Windows, and I was trying the compiled version of Monet under Linux. I have already started to compile Monet on Windows, but it's revealing to be a task harder than I thought... On Tue, Oct 8, 2013 at 12:08 PM, Martin Kersten <Martin.Kersten@cwi.nl>wrote:
On 10/8/13 12:16 PM, Pedro Salgueiro wrote:
Unfortunately we can't provider specific details about the application.
Basically, it create tables on-demand, some of them(very few) may have at most a few hundreds of columns. Among these tables, some are temporary and a very short life-time, being dropped when they are no longer needed. We really had a bug which was preventing some temporary tables from being deleted. Still, I really doubt that we have reached the limit of 16M columns. This is the main reason why we wanted to recover the database, so we can understand what the problem was.
We are dropping the temporary tables with a simple "drop table" sql statement. Could this method of dropping the tables leave some "garbage" behind, affecting the number of columns seen by Monet, even when the tables has already been deleted? Is there the need of doing some cleanup after dropping a table? If so, what should we do?
If you are expecting left-over garbage from the application, then look into the SQL catalog tables for their information. If you expect low level garbage, go and look in the growth rate of DBfarm/bat* directory. regards, Martin
Best regards, Pedro
On Mon, Oct 7, 2013 at 8:16 PM, Martin Kersten <Martin.Kersten@cwi.nl<mailto: Martin.Kersten@cwi.nl>**> wrote:
Hi
Perhaps you can share more information about your application. Is it product we can refer too? Can you give an url?
Having a database grow beyond 16M columns is indicative of some series application design issues, e.g. proper garbage collection of spurious tables left behind by your application as persistent. Given your indicative size of the database (1.1G) it appears that otherwise columns will be in the order of a few MB each.
regards, Martin
On 10/7/13 7:23 PM, Pedro Salgueiro wrote:
I changed the variables one at a time, never both at the same time. Still, I get the other error while connecting to the same DB.
I used the source obtained via "apt-get source monetdb5-server monetdb5-sql monetdb-client". After changing the code, I cleaned up everything with "make clean", then I recompiled everything using "dpkg-buildpackage" which build the new packages. I guess this should recompile everything, right?
I might try a new build from the source tarball to see if I get the same behavior with this DB.
Best regards, Pedro Salgueiro
On Mon, Oct 7, 2013 at 4:52 PM, Sjoerd Mullender <sjoerd@acm.org<mailto: sjoerd@acm.org> <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org>>> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When you change either one of those variables, you have to make absolute sure that you recompile *everything*. You only need to change one. The total number of BATs allowed in the system is (1 << BBPINITLOG) * N_BBPINIT. BAT descriptors get allocated (1 << BBPINITLOG) at a time and there are a maximum of N_BBPINIT of such allocations (I guess this is the implication: the maximum number of internal BAT descriptors is raised at the cost of memory for those descriptors; you choose how fast the number of allocated descriptors grows by choosing which constant you modify). I don't think there is a dependency in the dbfarm on either of those variables. Since we've never tried to recover from the error you faced, I don't know whether the new error is related or not.
On 2013-10-07 15:57, Pedro Salgueiro wrote: > I have already to changed the value of both variables, on the > Feb2013-SP4 version. Now I get a different error when the client > tries to connect to the same database: > > 2013-10-07 14:35:46 MSG merovingian[8219]: database 'testbd' > (8278) has exited with exit status 15 2013-10-07 14:35:46 ERR > merovingian[8219]: database 'testbd' crashed after SIGTERM > 2013-10-07 14:35:46 ERR merovingian[8219]: client error: database > 'testbd' did not initialise the sql scenario > > > Is this a completely different error or is it related with the > initial problem? > > I wonder if the database got corrupted when the initial problem > occurred. If the database got corrupted, is there any way to get it > back to normal operation? > > Also, what are the implications of changing these two variables? > Would this increase the memory consumption or harm the performance > of MonetDB in some way? > > Thank you, Pedro Salgueiro > > > > On Mon, Oct 7, 2013 at 1:52 PM, Sjoerd Mullender < sjoerd@acm.org <mailto:sjoerd@acm.org> <mailto:sjoerd@acm.org <mailto: sjoerd@acm.org>> > <mailto:sjoerd@acm.org <mailto:sjoerd@acm.org> <mailto: sjoerd@acm.org <mailto:sjoerd@acm.org>>>> wrote: > > The message means that you are creating too many BATs, probably > because you're using too many tables with too many columns. The > number of BATs is a hard limit, but it can easily be extended with > a recompilation of MonetDB. Increase one of the numbers N_BBPINIT > or BBPINITLOG in gdk/gdk.h and recompile everything. Alternatively, > rethink your database schema and use fewer tables and columns. > > On 2013-10-07 13:02, Pedro Salgueiro wrote: >> Hi, > >> We have a MonetDB instance which was working without problems >> and suddenly stopped working. The following error shows up in >> "merovingian.log" when the client(mclient) tries to connect to >> the database: > >> 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying >> to extend BAT pool beyond the limit (16384000) 2013-10-07 >> 10:35:27 ERR testbd[12111]: 2013-10-07 10:35:27 MSG >> merovingian[12015]: database 'testbd' (12111) has exited with >> exit status 1 2013-10-07 10:35:27 ERR merovingian[12015]: client >> error: database 'testbd' has crashed after starting, manual >> intervention needed, check monetdbd's logfile for details >> 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has >> crashed after start on 2013-10-02 16:36:06, attempting restart, >> up min/avg/max: 0s/0s/0s, crash average: 1.00 1.00 0.83 >> (25-0=25) > > >> The DB size is about 1.1 GB. I really don't know how many >> tables, columns or records this specific database has, as it was >> being used for a while and the data is created/uploaded daily by >> its users. > >> I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a >> few question about this issue: > >> 1 - Is there any way to see a more detailed log about the error? > >> 2 - Is it possible to recover the DB from this error? I saw >> somewhere in the mailing list that removing the sql_logs could >> help starting the DB, at the cost of loosing some data. Already >> tried that with no success. > >> 3 - What is this "BAT pool limit" thing? Is this related in >> some way with the number of tables, number of columns per table, >> or total amount of columns that a DB can have? > >> Thank you, Pedro Salgueiro > > > > > >> ______________________________**___________________ users-list >> mailing list users-list@monetdb.org <mailto: users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**>__> >> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**>__>>
>> https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
> > > > ______________________________**___________________ users-list mailing > list users-list@monetdb.org <mailto: users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**>__> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org**>__>>
> https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
> > > > > ______________________________**___________________ users-list mailing > list users-list@monetdb.org <mailto: users-list@monetdb.org**> <mailto:users-list@monetdb.org <mailto: users-list@monetdb.org**>__> > https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
>
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUlLYwD7g04AjvIQpAQIXuw**__QAqsq/1uEzVi2UNXSFOBC+__** ZW0dn5VjfYbi 8UcQr2Hb22iODA+5Xdat3hj04ZVqB/**__** 8UgE4P68MzAC23HusltxM6AZYhD4uF**__AJY3 /__**SzLNEBlKF2C3BqnIKyWlSqjtOwOArc**__RJC5RdGSd7Fj/** 48UVCrR0HtWVqhT4/__O9Q
LxR5iDsoUko= =Y/wp -----END PGP SIGNATURE----- ______________________________**___________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org**> <mailto: users-list@monetdb.org <mailto:users-list@monetdb.org**>__> https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**___________________
users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org**> https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**___________________
users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org**> https://www.monetdb.org/__**mailman/listinfo/users-list<https://www.monetdb.org/__mailman/listinfo/users-list><; https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**_________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
______________________________**_________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/**mailman/listinfo/users-list<https://www.monetdb.org/mailman/listinfo/users-list>
participants (3)
-
Martin Kersten -
Pedro Salgueiro -
Sjoerd Mullender