test_rename_drop continues to be flaky in non-HDFS environment. HMS
seems to be slower to response in non-HDFS environment. There are also
possibillity of table lock contention with background TableLoader and
CatalogDelta thread that cause DROP query happen after RENAME instead of
BEFORE.
This patch limit running test_rename_drop in HDFS and exhaustive mode in
the meantime.
Change-Id: Ie55e6d6093367c454cf3e31ed8a409b6e091193d
Reviewed-on: http://gerrit.cloudera.org:8080/22933
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Michael Smith <michael.smith@cloudera.com>
test_rename_drop fail with NPE after IMPALA-14042. This is because
CatalogServiceCatalog.renameTable() return null for not finding the
database of oldTableName. This patch change renameTable() to return
Pair.create(null, null) for that scenario.
Refactor test_rename_drop slightly to ensure that invalidating the
renamed table and dropping it are successful.
Testing:
- Add checkNotNull precondition in
CatalogOpExecutor.alterTableOrViewRename()
- Increase catalogd_table_rename_delay delay to 6s to ensure that DROP
query happen in Catalogd before renameTable() called. Manually
observed that No NPE is shown anymore.
Change-Id: I7a421a71cf3703290645e85180de8e9d5e86368a
Reviewed-on: http://gerrit.cloudera.org:8080/22899
Reviewed-by: Quanlong Huang <huangquanlong@gmail.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
TestConcurrentRename.test_rename_drop has been flaky because the
INVALIDATE query may arrive ahead of the ALTER TABLE RENAME query. This
patch deflake it by changing the sleep with admission control wait and
catalog version check. The first INVALIDATE query will only start after
catalog version increase since CREATE TABLE query.
Testing:
Loop the test 50x and pass them all.
Change-Id: I2539d5755aae6d375400b9a1289a658d0e7ba888
Reviewed-on: http://gerrit.cloudera.org:8080/22876
Reviewed-by: Yida Wu <wydbaggio000@gmail.com>
Reviewed-by: Jason Fehr <jfehr@cloudera.com>
Tested-by: Riza Suminto <riza.suminto@cloudera.com>
Handles the error "Table/view rename succeeded in the Hive Metastore,
but failed in Impala's Catalog Server" rather than failing the table
rename. This error happens when catalog state catches up to the alter
event from our alter_table RPC to HMS before we renameTable explicitly
in the catalog. The catalog can update independently due to a concurrent
'invalidate metadata' call.
In that case we use the oldTbl definition we already have - updated from
the delete log if possible - and fetch the new table definition with
invalidateTable to continue, automating most of the work that the error
message suggested users do via 'invalidate metadata <tbl>'.
Updated the test_concurrent_ddls test to remove handle_rename_failure
and ran the tests a dozen times. Adds concurrency tests with
simultaneous rename and invalidate metadata that previously would fail.
Change-Id: Ic2a276b6e5ceb35b7f3ce788cc47052387ae8980
Reviewed-on: http://gerrit.cloudera.org:8080/22807
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
