I am querying Hadoop using Hive and trying to return back data from a SELECT statement as a series of measurements.
The select can take quite a long time, especially if the cluster is busy.
I was thinking of using the 'org.rhq.plugins.database.CustomTableComponent' to perform the query, but the query may not be appropriate for the 'getValues' method. Is there a suggested approach?
One would be to adjust the database plugin to store the results of the query in a separate timer thread. This could be enabled or disabled by configuration. Should I make this change to the database plugin?
> One would be to adjust the database plugin to store the results of the query in a separate timer thread.
> This could be enabled or disabled by configuration. Should I make this change to the database plugin?
That's that way I would think you would have to do it. This is analogous to what we did with the asynchronous availability collector (we had the same problem when collecting availability - some resources took too long to even respond to what is supposed to be a quick avail check). See : http://rhq-project.org/display/RHQ/Design-Asynchronous+Availability+Collector and BZ 536173 for how we solved that. We wrote some utility class to do it (so any plugin can reuse the functionality) and plugins are free to incorporate it to pick up that ability to run avail checks asynchronously.
Looking at the implementation, I'm not sure why you are using Executor not ScheduledExecutorService for the availability checks. Basically you've reinvented the wheel (timer) here. It's probably not the end of the world to create a dozen or so threads but not really sensible either.
The benefit with ScheduledExecutorService is that it can also perform thread interruption using FutureTask.cancel(true) which plugins will probably want to use.
The biggest impact would be to the "ResourceContext" class constructor, but shouldn't impact plugins at all as ScheduledExecutorService extends Executor.
AvailabilityCollectorRunnable should probably be changed to use this as well.