IMPALA-14349: Encode FileDescriptors in time in loading Iceberg Tables

With this patch we create Iceberg file descriptors from
LocatedFileStatus objects during IcebergFileMetadataLoader's
parallelListing(). This has the following benefits:
 * We parallelize the creation of Iceberg file descriptor objects
 * We don't need to maintain a large hash map with all the
   LocatedFileStatus objects at once. Now we only need to keep a few
   LocatedFileStatus objects per partition in memory while we are
   converting them to Iceberg file descriptors. I.e., the GC is free to
   destroy the LocatedFileStatus objects we don't use anymore.

This patch retires startup flag 'iceberg_reload_new_files_threshold'.
Since IMPALA-13254 we only list partitions that have new data files,
and we load them in parallel, i.e. efficient incremental table loading
is already covered. From that point the startup flag only added
unnecessary code complexity.

Measurements

I created two tables (from tpcds.store_sales) to measure table loading
times for large tables:

Table #1:
  PARTITIONED BY SPEC(ss_item_sk, BUCKET(5, ss_sold_time_sk))
  partitions: 107818
  files: 754726

Table #2:
  PARTITIONED BY SPEC(ss_item_sk)
  partitions: 18000
  files: 504224

Time taken in IcebergFileMetadataLoader.load() during full table reload:
+----------+-------+------+---------+
|          | Base  | New  | Speedup |
+----------+-------+------+---------+
| Table #1 | 17.3s | 8.1s |    2.14 |
| Table #2 |  7.8s | 4.3s |     1.8 |
+----------+-------+------+---------+

I measured incremental table loading only for Table #2 (since there are
more files per partition this is the worse scenario for the new code, as
it only uses file listings, and each new file were created in a separate
partition)

Time taken in IcebergFileMetadataLoader.load() during incremental table
reload:
+------------+------+------+---------+
| #new files | Base | New  | Speedup |
+------------+------+------+---------+
|          1 | 1.4s | 1.6s |     0.9 |
|        100 | 1.5s | 1.9s |     0.8 |
|        200 | 1.5s | 1.5s |       1 |
+------------+------+------+---------+

We lose a few tenths of a second, but I think the simplified code
justifies it.

Testing:
 * some tests were updated because we we don't have
   startup flag 'iceberg_reload_new_files_threshold' anymore

Change-Id: Ia1c2a7119d76db7ce7c43caec2ccb122a014851b
Reviewed-on: http://gerrit.cloudera.org:8080/23363
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>

common/thrift/BackendGflags.thrift

@@ -260,8 +260,6 @@ struct TBackendGflags {
 
   114: required bool use_jamm_weigher
 
-  115: required i32 iceberg_reload_new_files_threshold
-
   116: required bool enable_skipping_older_events;
 
   117: required bool enable_json_scanner