1
0
mirror of synced 2025-12-31 15:03:11 -05:00
Files
airbyte/airbyte-db
Davin Chia 8a0d12fc80 Add migrations to support progress bar. (#19191)
Follow up to #18953.

Implement all the DB migrations required for a progress bar.

The main change here is to support saving:

the estimated records/bytes at the sync level
the estimated records/bytes and emitted records/bytes at the stream level
After this, I'll put up a PR for the persistence layer changes, which will writing to and reading from these columns.

Finally, I'll wire this into the API changes, which are currently stubs.

- add the estimated_records and estimated_bytes columns to the SyncStats table.
- create a stream_stats table
  - estimated and emitted records/bytes column
  - contains attempt_id and stream_name columns. Unique constraints on these two columns.
  - foreign key to the attempt_id table.
  - this table hopefully sets us up for the parallel sync work.
2022-11-10 22:35:15 -08:00
..