jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-23 21:08:39 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-10613: (addendum) Fix test on S3 builds

Browse Source 
The test_metastore_service.py fails on S3 builds because it expects
the filemetadata's object dictionary to be present. However, if the
table is located on S3 then there are no file-blocks in the returned
file-metadata and hence the length of obj_dict will be 0.

This patch fixes the test in S3 builds by not asserting the check
on S3, ADLS, GCS builds.

Testing Done:
1. Ran the test locally to confirm it is working.
2. [WIP] Ran the test on a S3 environment where tables are created
on S3.

Change-Id: I6ac291529dc0661abdfc2d4f48924a2c4b807c40
Reviewed-on: http://gerrit.cloudera.org:8080/17483
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Vihang Karajgaonkar <vihang@cloudera.com>
This commit is contained in:
Vihang Karajgaonkar
2021-05-21 13:13:33 -07:00
committed by Vihang Karajgaonkar
parent b50d60a6c5
commit 8087c75f62
1 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
tests/custom_cluster/test_metastore_service.py
View File

@@ -27,6 +27,7 @@ from hive_metastore.ttypes import SerDeInfo
from tests.util.event_processor_utils import EventProcessorUtils
from tests.common.custom_cluster_test_suite import CustomClusterTestSuite
from tests.common.impala_test_suite import ImpalaTestSuite
from tests.util.filesystem_utils import (IS_S3, IS_ADLS, IS_GCS)
class TestMetastoreService(CustomClusterTestSuite):
@@ -596,7 +597,10 @@ class TestMetastoreService(CustomClusterTestSuite):
test_part_names = list(part_names)
if expect_files:
assert get_parts_by_names_result.dictionary is not None
assert len(get_parts_by_names_result.dictionary.values) > 0
# obj_dict will only be populated when the table is on HDFS
# where block locations are available.
if not IS_S3 and not IS_GCS and not IS_ADLS:
assert len(get_parts_by_names_result.dictionary.values) > 0
else:
assert get_parts_by_names_result.dictionary is None
partitions = get_parts_by_names_result.partitions
@@ -621,7 +625,10 @@ class TestMetastoreService(CustomClusterTestSuite):
assert filemetadata is not None
assert filemetadata.data is not None
assert obj_dict is not None
assert len(obj_dict.values) > 0
# obj_dict will only be populated when the table is on HDFS
# where block locations are available.
if not IS_S3 and not IS_GCS and not IS_ADLS:
assert len(obj_dict.values) > 0
def __assert_no_filemd(self, filemetadata, obj_dict):
"""