Weird concurrency issue with JDBC getConnection
Dear Monet Team, I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang. I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this. Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Dear Percy, Thanks for the information. We'll keep an eye on this issue. regards, Martin On 1/9/13 10:04 PM, Percy Wegmann wrote:
Dear Monet Team,
I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang.
I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this.
Cheers,
--
Percy Wegmann +1 512 637 8500 ext 148
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Hi Percy, On 09-01-2013 15:04:17 -0600, Percy Wegmann wrote:
Dear Monet Team,
I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang.
I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this.
If would be useful to know what the threads are hanging on when you feel they are stuck. I suspect that the server doesn't like you bombarding it in parallel. The code you reference is all pretty object oriented, and not using any shared resources. -- Fabian Groffen fabian@monetdb.org column-store pioneer http://www.monetdb.org/Home _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
I believe that your hypothesis is right. I created the attached two versions of a test to reproduce this. One works, one doesn't. The only difference between them is that the working one synchronizes on the shared counter object. The failing test always gets through about 16-18 connects before hanging. All of the hung threads are hanging in the same place, on SocketInputStream.socketRead, which was ultimately called from MapiSocket.connect(). Below is a copy of the stack. I'm running the 11.13.7 release with the 2.8 JDBC driver, server on Ubuntu Linux, client on MacOS. The database to which I'm connecting is configured to accept 64 connections. Cheers, Percy Thread [Thread-28] (Suspended) owns: BufferedInputStream (id=78) owns: InputStreamReader (id=79) SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int) line: not available [native method] SocketInputStream.read(byte[], int, int, int) line: 150 SocketInputStream.read(byte[], int, int) line: 121 BufferedInputStream.fill() line: 235 BufferedInputStream.read1(byte[], int, int) line: 275 BufferedInputStream.read(byte[], int, int) line: 334 MapiSocket$BlockInputStream._read(byte[], int) line: 846 MapiSocket$BlockInputStream.readBlock() line: 895 MapiSocket$BlockInputStream.read(byte[], int, int) line: 965 StreamDecoder.readBytes() line: 283 StreamDecoder.implRead(char[], int, int) line: 325 StreamDecoder.read(char[], int, int) line: 177 InputStreamReader.read(char[], int, int) line: 184 BufferedMCLReader(BufferedReader).fill() line: 154 BufferedMCLReader(BufferedReader).readLine(boolean) line: 317 BufferedMCLReader(BufferedReader).readLine() line: 382 BufferedMCLReader.readLine() line: 119 MapiSocket.connect(String, int, String, String, boolean) line: 275 MapiSocket.connect(String, int, String, String) line: 248 MonetConnection.<init>(Properties) line: 232 MonetDriver.connect(String, Properties) line: 171 DriverManager.getConnection(String, Properties, ClassLoader) line: 579 DriverManager.getConnection(String, String, String) line: 221 GetConnectionTest$DoGetConnection.run() line: 37 Thread.run() line: 722 On Thu, Jan 10, 2013 at 3:33 AM, Fabian Groffen <fabian@monetdb.org> wrote:
Hi Percy,
On 09-01-2013 15:04:17 -0600, Percy Wegmann wrote:
Dear Monet Team,
I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang.
I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this.
If would be useful to know what the threads are hanging on when you feel they are stuck. I suspect that the server doesn't like you bombarding it in parallel. The code you reference is all pretty object oriented, and not using any shared resources.
-- Fabian Groffen fabian@monetdb.org column-store pioneer http://www.monetdb.org/Home
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
-- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Oh, please also note that the tests behave the same if we do not bother to close the connection. On Thu, Jan 10, 2013 at 8:09 AM, Percy Wegmann <percy@clicksecurity.com>wrote:
I believe that your hypothesis is right.
I created the attached two versions of a test to reproduce this. One works, one doesn't. The only difference between them is that the working one synchronizes on the shared counter object.
The failing test always gets through about 16-18 connects before hanging.
All of the hung threads are hanging in the same place, on SocketInputStream.socketRead, which was ultimately called from MapiSocket.connect(). Below is a copy of the stack.
I'm running the 11.13.7 release with the 2.8 JDBC driver, server on Ubuntu Linux, client on MacOS. The database to which I'm connecting is configured to accept 64 connections.
Cheers, Percy
Thread [Thread-28] (Suspended) owns: BufferedInputStream (id=78) owns: InputStreamReader (id=79) SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int) line: not available [native method] SocketInputStream.read(byte[], int, int, int) line: 150 SocketInputStream.read(byte[], int, int) line: 121 BufferedInputStream.fill() line: 235 BufferedInputStream.read1(byte[], int, int) line: 275 BufferedInputStream.read(byte[], int, int) line: 334 MapiSocket$BlockInputStream._read(byte[], int) line: 846 MapiSocket$BlockInputStream.readBlock() line: 895 MapiSocket$BlockInputStream.read(byte[], int, int) line: 965 StreamDecoder.readBytes() line: 283 StreamDecoder.implRead(char[], int, int) line: 325 StreamDecoder.read(char[], int, int) line: 177 InputStreamReader.read(char[], int, int) line: 184 BufferedMCLReader(BufferedReader).fill() line: 154 BufferedMCLReader(BufferedReader).readLine(boolean) line: 317 BufferedMCLReader(BufferedReader).readLine() line: 382 BufferedMCLReader.readLine() line: 119 MapiSocket.connect(String, int, String, String, boolean) line: 275 MapiSocket.connect(String, int, String, String) line: 248 MonetConnection.<init>(Properties) line: 232 MonetDriver.connect(String, Properties) line: 171 DriverManager.getConnection(String, Properties, ClassLoader) line: 579 DriverManager.getConnection(String, String, String) line: 221 GetConnectionTest$DoGetConnection.run() line: 37 Thread.run() line: 722
On Thu, Jan 10, 2013 at 3:33 AM, Fabian Groffen <fabian@monetdb.org>wrote:
Hi Percy,
On 09-01-2013 15:04:17 -0600, Percy Wegmann wrote:
Dear Monet Team,
I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang.
I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this.
If would be useful to know what the threads are hanging on when you feel they are stuck. I suspect that the server doesn't like you bombarding it in parallel. The code you reference is all pretty object oriented, and not using any shared resources.
-- Fabian Groffen fabian@monetdb.org column-store pioneer http://www.monetdb.org/Home
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
--
Percy Wegmann +1 512 637 8500 ext 148
-- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
participants (3)
-
Fabian Groffen -
Martin Kersten -
Percy Wegmann