Joe McDonnell 938be0e840 IMPALA-7738: Implement timeouts for HDFS open calls ...

This is part 1 of a push to add timeouts for all HDFS operations.
It adds timeouts for opening an HDFS file handle.

It introduces a new SynchronousThreadPool, which executes
an operation in a thread pool and waits up to a specified
timeout for the operation to complete. This type of thread
pool can accept any subclass of SynchronousWorkItem, and
a single thread pool can process different types of work
items. It is tested by a new test case in thread-pool-test.

This also introduces a new HdfsMonitor which implements
timeouts for HDFS operations, currently limited to
hdfsOpenFile(). This is implemented using a SynchronousThreadPool.
The timeout for hdfs operations is specified by
hdfs_operation_timeout_sec, which defaults to 5 minutes.

Testing:
1. Added a test to thread-pool-test for the new
   SynchronousThreadPool.
2. Core tests
3. Added a custom cluster test that does "kill -STOP"
   for the NameNode and verifies that a subsequent
   hdfsOpenFile operation times out.

Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454
Reviewed-on: http://gerrit.cloudera.org:8080/11874
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>

2018-12-07 03:50:11 +00:00

18 KiB

Executable File

Raw Blame History

View Raw