-
1. Re: Teiid join threading question
rareddy Nov 10, 2015 6:35 PM (in response to kchen007)Possibly your resource-adapter's ManagedConnectionFactory and ResourceAdapter class's equals and hashcode method may not have implemented correctly.
-
2. Re: Teiid join threading question
shawkins Nov 10, 2015 6:38 PM (in response to kchen007)> My understanding is that Teiid will get a different connections from JBOSS' connection pool.
Yes, concurrent source queries will use different connections from the pool. If you have the same connection in use at the same time for concurrent source queries, that is not expected. Is it possible that the resource adapter is providing the same physical connection multiple times to the connection pooling logic?
> Is there any way to get around it? also what is the best way to pinpoint where exactly the association is wrong.
You can prevent concurrent source access altogether by just executing within the scope of a transaction, which will force the engine to serialize the plan.
In general if you are getting the same connection concurrently, that's not expected behavior and you'd likely need to have logging at the AS JCA/pooling layer at a debug level or higher and/or debug the resource adapter.
-
3. Re: Teiid join threading question
kchen007 Nov 12, 2015 4:50 PM (in response to shawkins)Some more finding, this is happening when we insert a new entity, after that we tried to read the content back to fill auto-generated values.
Our first attempt is to use JPA refresh() to read it back, but the primary key is system generated, the value is null; so the refresh() will not return anything. When that happened, it appears that Teiid will return all the connections to connection pool used by the worker thread for the refresh() call or something like that, even the worker thread has not finished; see the 2nd picture, the worker thread are waiting for result, but the connection has been closed and mc=null. when I looked at the actual physical connection from the stacktrace, it is the same as the physical connection used in the 1st picture, another worker thread.
when the refresh() failed, we will try to read the entity back via alternative key columns(which are non-null during insertion), at that time one of the worker will pick up the returned connection(see 1st picture) which is still used in un-returned worker thread from refresh().
this is much easy to reproduce when I associate the process with single processor only.
-
4. Re: Teiid join threading question
kchen007 Nov 12, 2015 5:46 PM (in response to kchen007)here are the 2 screen shots that shows one thread is closing the connection that is still running by planner. so, I guess the planner should not close unfinished worker, let them finish and close by themselves.
this one shows it is closing the connection for worker #21447 from thread Worker57_QPQ3559.
this one shows the worker #21447 is still running
-
5. Re: Teiid join threading question
shawkins Nov 12, 2015 5:56 PM (in response to kchen007)I think you are seeing a regression from TEIID-3275. The fully close source is first canceling the task, which will make it immediately report isDone true, which means that it will immediately run the completion listener. That is not the intent as we want the closure to happen after the full affect of the cancel.
-
6. Re: Teiid join threading question
shawkins Nov 12, 2015 6:45 PM (in response to shawkins)Actually I may have spoken too soon, while the cancel logic does need adjusting the ConnectorWorkItem is fully synchronized on the execute, more, and close methods. This means that the close will not happen until after the more call is finished and after that the work item will not be usable. So I don't think this would explain what you are seeing.
-
7. Re: Teiid join threading question
shawkins Nov 13, 2015 7:29 AM (in response to shawkins)What version are you on?
-
-
9. Re: Teiid join threading question
shawkins Nov 13, 2015 10:20 AM (in response to kchen007)This was likely addressed by [TEIID-3612] NPE in connectorworkitem after close - JBoss Issue Tracker as even if close is called when there is still pending work, ConnectoWorkItem.more will not proceed.
-
10. Re: Teiid join threading question
kchen007 Nov 13, 2015 11:42 AM (in response to shawkins)that is not the case here. when it happened, the execution is not null. also the connection object(type org.teiid.resource.spi.WrappedConnection) is also not null, but it is state has been changed somewhere else, with close=true and mc=null.
-
11. Re: Teiid join threading question
shawkins Nov 13, 2015 11:56 AM (in response to kchen007)Can you try this with 8.12 or apply the patch to 8.7?