This takes steps to make Python 2 behave like Python 3 as
a way to flush out issues with running on Python 3. Specifically,
it handles two main differences:
1. Python 3 requires absolute imports within packages. This
can be emulated via "from __future__ import absolute_import"
2. Python 3 changed division to "true" division that doesn't
round to an integer. This can be emulated via
"from __future__ import division"
This changes all Python files to add imports for absolute_import
and division. For completeness, this also includes print_function in the
import.
I scrutinized each old-division location and converted some locations
to use the integer division '//' operator if it needed an integer
result (e.g. for indices, counts of records, etc). Some code was also using
relative imports and needed to be adjusted to handle absolute_import.
This fixes all Pylint warnings about no-absolute-import and old-division,
and these warnings are now banned.
Testing:
- Ran core tests
Change-Id: Idb0fcbd11f3e8791f5951c4944be44fb580e576b
Reviewed-on: http://gerrit.cloudera.org:8080/19588
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Joe McDonnell <joemcdonnell@cloudera.com>
Creating a single node plan for the following SQL sometime can slowdown,
with about hundreds of inlineviews to join, and view1, view2... outputs
hundreds of expressions.
select c1 from (select c1, id from view1 where c1 > 10) t1 join (select
c2, id from view2 where c1 > 10) t2 on t1.id = t2.id join ...
The reasons for the slow generation of plans are as follows
1. Many auxiliary predicates are added to GlobalState.conjuncts causing
performance degradation of Analyzer#getUnassignedConjuncts
2. In SingleNodePlanner#createInlineViewPlan the output smap is the
composition of the inline view's smap and the output smap of the inline
view's plan root. Multiple inline view joins cause
ExprSubstitutionMap#compose performance to degrade.
For 1, add GlobalState.conjunctsWithoutAuxExpr to save the registered
conjuncts without auxiliary predicate.
For 2, remove expressions from outputSmap that are not used according
to baseSmap.
Testing:
Add test tests/query_test/test_query_compilation.py
Repro query created single node plan went from 2.3 sec to 0.3 sec.
Change-Id: Ifb4011b6167a0e61438a73c4dba6f1cd0a4e8c6a
Reviewed-on: http://gerrit.cloudera.org:8080/17712
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Qifan Chen <qchen@cloudera.com>
